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(54) Optical code reader mouse 



(57) An optical code reader mouse device utilizes 
an imager, such as a CCD or CMOS imaging array to 
image printed indicia including symbols. The optical 
code reader may be utilized as a code reader, a video 
camera, to indicate relative movement between the de- 
vice and a surface, and lor other various imaging func- 
tions. Thus, the optical code reader may also have no 
internally moving parts utilized for sensing relative 
movement. The optical code reader may function simi lar 
to a conventional mouse, except that the relative motion 
is sensed by the imager. The optical code reader may 
detect the relative movement of artifacts across its field 
of view in any arbitrary direction within the local length 



setting of the device. The optical code reader may also 
be incorporated into a laptop and a pen based computer, 
and a PDA. Relative motion may be sensed when mov- 
ing either the housing or the surface itself, or both, rel- 
ative to the other. In the portable data processing em- 
bodiments, the optical code reader may also be utilized 
as a code reader a video camera, to indicate relative 
movement between the device and a surface, and for 
other various fringing functions. The surface may in- 
clude a finger, a hand, or the "pen" of the pen based 
computer, or almost any object within the field of view. 
Also, various modes and methods described herein may 
be utilized with the optical code reader. 
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Description 

Background of the Invention 

[0001] The present invention relates to a computer 
'mouse" device that is used to provide input to a com- 
puter, the mouse capable of providing relative tracking 
data with respect to a surface, and providing outputs 
representing the relative position of the mouse with re- 
spect to the surface. Further, the present invention re- 
lates to a mouse that is able to provide output of relative 
movement and also capable of capturing images, such 
as printed indicia and human readable text characters. 
The instant Application also incorporates by reference, 
as though fully set forth herein, the related Provisional 
Application No. 60/085,628 filed May 15, 1998, entitled 
1 PC-Mouse/I mager. 1 

Description of the Prior Art 

[0002] Prior art mechanisms for a computer mouse 
generally includes a ball element that is rotated when 
the mouse is moved across a surface such as a mouse 
pad. In addition, the prior art teaches a combined mouse 
and bar code scanner as taught in U.S. Patent Nos. 
5,448,050 and 5,814,804 to D. Kostizak, and assigned 
to the same assignee as herein. Both of these two pat- 
ents are incorporated by reference as though fully set 
forth herein. 

[0003] The prior art systems as disclosed in the Kos- 
tizak patents are described in relation to Figures 9 and 
10 of the instant Application. Referring to Figures 9 and 

10, there is shown a conventional integrated bar code 
reader and location detection apparatus having a shape 
and size of, and operative in a manner similar to, a com- 
puter mouse. Accordingly, the apparatus is referred to 
as a bar code reading mouse 10. The mouse 10 has a 
housing 12 that has top and bottom parts 14 and 16, 
that may snap together along a horizontal parting joint 
(not shown), The upper part of the housing 14 defines 
the top of the mouse 1 0 and the bottom part 1 6 defines 
the bottom of the mouse. The top has a pair of keys 18 
and 1 9, the location of which is shown by the dash line 
boxes in Figure 10. These keys operate electrical 
switches S\N^ and SW 2 as shown in Figure 11 . The bot- 
tom of the housing 1 2 has pads 20 that space the hous- 
ing from a surface 22 upon which barcodes appear. The 
spacing is such that a ball 24 of an incremental encoder 
26 contacts and engages the surface 22 and can roll 
universally as the mouse is moved in any direction along 
the surface 22. 

[0004] The incremental encoder 26 may be of the type 
conventionally used in computer mice. It provides an 
output indicating the relative displacement of the mouse 
along the surface 22. This displacement is converted by 
software in a computer terminal 28, as shown in Figure 

1 1 , so that the display screen 32 of the terminal 28 indi- 
cates a representation 30 of the location of the mouse. 



The prior Kostizak patent states that instead of a me- 
chanical encoder type mouse location detection means, 
an optical encoder may be used, in which use the sur- 
face 22 may be inscribed with orthogonal lines to which 
5 the optical encoder responds. 

[0005] Continuing with the description of Figures 
9-1 1 , reading of the bar codes is provided by a scan en- 
gine 34 having electrical and optical components 
mounted on a printed circuit board 36. This board 36 
also mounts the encoder 26 and supports the ball 24. 
Electrical circuit paths, or traces, and terminals from the 
electrical components, that are connected to the traces 
on the board 36, are not shown to simplify the illustra- 
tion. These circuits are connected to leads in a cable 38, 
that is a flexible cable extending from the mouse 1 0 to 
the computer terminal 28 of Figure 11. The computer 
terminal, or the board 36, may have components that 
provide an interface 40 that processes the output sig- 
nals generated in the mouse 10. The computer terminal 
may also provide input signals for controlling the oper- 
ation of the mouse 10, as will be explained hereinafter. 
The interface 40 may include signal conditioning means, 
decoders, and/or multiplexers of the type conventionally 
used to implement the interface functions. 
[0006] The scan engine 34 is of a design similar to the 
design of the scan engine described in U.S. Patent No. 
5,200,597 issued April 6, 1 993, by Jay M. Eastman, et 
al., and assigned to the same assignee as the present 
Application, and is incorporated by reference as though 
fully set forth herein. 

[0007] In the scan engine, a motor 42 pivots a mount 
44 having a reflective surface 46 that projects a scan- 
ning beam 48. This beam is generated by a laser diode 
in a barrel or mount 50, that reflects from a facet 52 of 
a light collecting mirror 54. The collected light is reflect- 
ed by the mirror 54 to a photo detector 55 and converted 
into analog bar code signals. These signals may be 
processed and decoded by circuits, including integrated 
circuit chips containing microprocessor devices and 
memory that are located on the board 36, but not shown 
in order to simplify the illustration. The board is mounted 
by mounts 37 on the inside surface of the bottom of the 
housing 12. When portable operation is desired, batter- 
ies can be carried on board the mouse. Radio, optical, 
e.g., infra-red, or acoustic links instead of wires may be 
used. 

[0008] The scan beam 48 is incident on a port mirror 
56 of a mirror assembly that includes knobs 58 extend- 
ing from the sides of the housing 1 2. The port mirror 56 
folds the scan beam 48 so that it is deflected downward- 
ly toward the surface 22 through either a front window 
or port 60 in the front end of the housing or a bottom 
window or port 62 in the bottom of the housing. For ex- 
ample, these ports may be covered by transparent 
plates, or instead may contain lenses for focusing the 
beam at the surface 22 or collimating the beam. 
[0009] In a first mode of operation, the mouse is 
tracked along the surface 22 to the proximity of a bar 
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code. Then SW 2 is actuated by operating the key 1 9 to 
provide an output through the interface to the terminal 
indicative of the location of the mouse in a scanning po- 
sition with respect to a selected code on the surface. 
Then SW 1 is actuated by the operating key 18 and the 
bar code scanner is enab led to provide a bar code signal 
output through the interface to the terminal, that proc- 
esses, and/or displays the output or a code correspond- 
ing thereto, or may store that code in a memory associ- 
ated with the computer in the terminal. 
[001 0] In another mode of operation of the device, the 
computer is programmed (information being stored in 
its memory/database) as to the position of barcodes on 
the surface to be read. Then outputs are provided by the 
computer in response to location information from the 
encoder 26 relayed to the computer via the interface 40. 
The computer generates an output that, alternatively to 
S Wl , enables the bar code scanner 34 to scan the code. 
Then only selected codes on the surface will be 
scanned. Also, the mouse may be lifted from the surface 
22, pointed at a bar code (e.g., on a package) and S\N^ 
operated to scan and read the bar code. Thus, the 
mouse is operative even when it is not placed on the 
surface 22 or another surface (e.g., the surface of the 
package). 

[0011] Moreover, the above described conventional 
integrated bar code reading and location mouse is able 
to read one dimensional bar codes placed upon a sur- 
face upon which the mouse operates. However, the con- 
ventional mouse is only able to lift, e.g., bar codes from 
surfaces The conventional mouse does not capture 
graphic data, for example, such as photographs or art, 
with the same imaging device that supplies the X-Y rel- 
ative movement data of the mouse to the computer dis- 
play. 

[0012] Several problems arise from the use of the 
aforementioned bar code reading and location mouse 
as well as all mice in general. First, typically, the incre- 
mental encoder 26 comprises a mechanical track ball 
that is highly prone to collecting dust, lint, dirt, and grime. 
As the mouse is operated, this dust, lint, dirt, and grime 
contaminates the inner workings of the incremental en- 
coder 26 and either causes it to intermittently malfunc- 
tion or to stop functioning entirely. The conventional 
mouse design and assembly also require a number of 
moving parts as well as a relatively large number of parts 
to assemble, thus, often decreasing its reliability while 
increasing its manufacturing cost. Also, a conventional 
mouse is not able to detect the relative movement of 
artifacts across its field of view, in any arbitrary direction 
within the desktop plane. 

[0013] One attempt to solve some of the problems 
with ball driven mouse designs is to utilize an infra-red 
("IR") Light Emitting Diode ("LED") for one axis, and a 
visible LED is utilized for a second axis of movement. A 
special pad must be provided with referenced grids 
printed upon it in order to provide a method to contrast 
the two wave lengths of light supplied by the IR and vis- 



ible LEDs. Thus, the mouse may only be used if the spe- 
cial pad is available. Unfortunately, even if the pad is 
available, another problem arises if the pad is rotated. 
If rotated from its preset orientation, the cursor display 
5 provided by the optical mouse will not move in the di- 
rection that a user might intuitively expect it to, i.e., as 
would a mouse utilizing a conventional track ball mech- 
anism. Instead, because the optical mouse senses 
movement relative to the "x" and "y" grid on the pad, a 
non-intuitive movement of the cursor on the display oc- 
curs, at any time the pad was rotated out of the pre- 
designated position. 

[001 4] Accordingly, it is an object of the instant inven- 
tion to provide an improved device and method of sens- 
ing relative motion that may reduce or eliminate the 
need or desire for a special pad, and further may be uti- 
lized to reduce or eliminate a non-intuitive cursor re- 
sponse when the surface is rotated relative to the 
mouse. 

[001 5] Another object of the instant invention is to pro- 
vide an improved device and method of sensing that 
may be utilized to recognize machine decodable sym- 
bols, such as one or two-dimensional bar codes. Anoth- 
er object of the instant invention is to provide an im- 
proved device and method of sensing that may be uti- 
lized to recognize text, so as to apply optical character 
recognition ("OCR") techniques. It is a further object of 
the invention is to provide an improved device and meth- 
od of sensing that may be utilized as a video system or 
video camera. 

[001 6] Yet another object of the invention is to provide 
an improved device and method of sensing that may be 
utilized to generate a mouse cursor function, wherein 
the mouse sensor is relatively stationary as compared 
to the surface. It is a further object of the invention to 
provide an improved device and method of sensing that 
may be utilized to generate a cursor signal intuitively 
similar to the user as if utilizing a portable computer 
touch pad device. Another object of the instant invention 
is to provide an improved device and method of sensing 
wherein the relatively stationary mouse sensor may be 
utilized as a video conference camera or video system. 
[0017] It is a further object of the invention to provide 
an improved device and method of sensing that may be 
utilized as a mouse that has an adjustable lens function 
for altering or adjusting the focal length of the mouse so 
that the mouse may be used for a plurality of imaging 
functions comprising at least some of those functions 
described herein. It is another object of the invention to 
provide an improved device and method of sensing that 
may be utilized with an automatic or software controlled 
variable focus lens assembly. 

[0018] It is another object of the invention to provide 
an improved device and method of sensing that may be 
utilized to provide the user with a visual representation 
of the area to be imaged by the mouse. 
[0019] Finally, an object of the present invention is to 
provide a solution for at least some of these aforemen- 
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lioned problems. 
Summary of the Invention 

[0020] Briefly described, the present invention pro- 5 
vides lor a computer imaging mouse device that utilizes 
a CMOS imaging array to both monitor a relative move- 
ment of the mouse with respect to a surface as well as 
capture images, such as printed indicia. The computer 
imaging mouse preferably has no internally moving 10 
parts such as a mouse ball or a mouse ball mechanism 
for sensing an X-Y displacement. 
[0021] The imager mouse provides an X-Y displace- 
ment indication for use on a computer display similar to 
a conventional mouse, except that the motion induced fS 
by the operator is sensed by a CMOS imaging array that 
detects relative movement of artifacts across its field of 
view, in any arbitrary direction within the focal length set- 
ting of the imager mouse. The imager mouse can be 
used to capture printed indicia, such as bar codes, 20 
stacked bar codes, matrix codes or other printed sym- 
bols which are machine decodable. However, the imag- 
er mouse can also be used as a scanner, capable of 
incrementally reading printed symbols extending be- 
yond the immediate field of view of an imager mouse 2$ 
window. If either the imager mouse and a surface are 
moved relative to each other, images may be captured 
in real time, passed to the host computer, and then as- 
sembled into an electronic representation of the 
scanned image, while using directional displacement 30 
cues from the mouse. 

Brief Description ol the Drawings 

[0022] The invention is described in detail herein with 3S 
reference to the drawings in which: 



Figure 3B illustrates a top view of another code 
reading and mouse device resulting from the prac- 
tice of the invention; 

Figure 3C illustrates a cross-sectional top view of 
another code reading and mouse device resulting 
from the practice ol the invention, the section being 
taken along the line 3C-3C in Figure 3A; 
Figure 3D illustrates a top view of another code 
reading and mouse device resulting from the prac- 
tice of the invention; 

Figure 4A is a top view of a symbol that may be uti- 
lized in the practice of the invention; 
Figures 4B-C illustrate a method of capture of an- 
other symbol that may be utilized in the practice of 
the invention; 

Figure 4D illustrate another method of capture of a 
symbol that may be utilized in the practice of the 
invention; 

Figure 5A is a cross-sectional side view of a lens 
device that may be utilized in the practice of the in- 
vention, the section being taken along the line 5A- 
5A in Figure 5B; 

Figure 5B is a front view of the lens device of Figure 
5A that may be utilized in the practice of the inven- 
tion; 

Figure 6 is a schematic block diagram of another 
arrangement of the device, as shown in Figures 1 -3, 
that may be utilized in the practice of the invention; 
Figures 7A-B are block diagrams of alternate ar- 
rangements that may be utilized in the practice of 
the invention; 

Figure 8 is a block diagram of another alternate ar- 
rangement that may be utilized in the practice of the 
invention; 

Figure 9 is a cross-sectional side view of an inte- 
grated bar code reading and location mouse device, 
the section being taken along the line 9-9 in Figure 
10; 

Figu re 1 0 is a top view of the device shown in Figu re 
9 with the top of the housing removed to illustrate 
the reading and location components therein; 
Figure 11 is a schematic block diagram of a typical 
installation of the device shown in Figures 9 and 10; 

[0023] The accompanying drawings, wherein like nu- 
merals denote like elements, are incorporated into and 
constitute a part of the specification, and illustrate pres- 
ently preferred exemplary embodiments of the inven- 
tion. The drawings, together with the general description 
given above, and in the detailed description of the pre- 
ferred embodiments given below, serve to explain the 
principles of the invention. 

Detailed Description of the Preferred Embodiment 

[0024] The instant invention is illustrated utilizing a 
computer mouse device. Figures 1-8 illustrate-various 
techniques and alternatives in the practice of the current 



Figure 1 A illustrates a side view of a code reading 
and mouse device resulting from the practice of the 
invention; *o 
Figure 1 B illustrates a top view of a code reading 
and mouse device resulting from the practice of the 
invention; 

Figure 1 C illustrates a cross-sectional top view of a 
code reading and mouse device resulting from the 45 
practice of the invention, the section being taken 
along the line 1C-1C in Figure 1A; 
Figure 2A illustrates a side view of another mouse 
device resulting from the practice of the invention; 
Figure 2B illustrates a top view of another mouse so 
device resulting from the practice of the invention; 
Figure 2C illustrates a cross-sectional top view of 
another mouse device resulting from the practice of 
the invention, the section being taken along the line 
2C-2C in Figure 2A; ss 
Figure 3A illustrates a side view of another code 
reading and mouse device resulting from the prac- 
tice of the invention; 
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invention in the following description p the same refer- 
ence numerals are utilized for substantially similar ele- 
ments in Figures 1-11, for the purpose of clarity. How- 
ever, it will be apparent to one skilled in the art, for ex- 
ample, that some of the "like" elements may not actually 
need to be substantially similar or identical in order to 
practice the invention. Also, embodiments of the inven- 
tion, as one skilled in the an" recognizes, may be prac- 
ticed on devices with these variations, and differences, 
as well as on other alternate embodiments, and all are 
within the scope of this description and the appended 
claims. 

[0025] Further, some of the bus and signal lines uti- 
lized in the practice of certain arrangements of the in- 
stant invention are shown in Figures 1-8. However, it will 
be understood by one skilled in the art that generally the 
electrical circuit paths, traces, terminals, and electrical 
components, and respective software components, that 
may comprise at least portions of the various arrange- 
ments of the instant invention as illustrated in Figures 
1 -8, are not shown, in order to simplify the illustrations. 
[0026] Figu res 1 A-C illustrate an arrangement of a de- 
vice in the practice of the instant invention, namely a 
computer mouse device 100. Figure 1 A illustrates a side 
view of the mouse 100, and Figure 1 B illustrates a top 
view of the computer mouse 100 of Figure 2A. Figure 
1 C illustrates a cross-sectional top view of the computer 
mouse 100 of Figure 2A, the section being taken along 
the line 1C-1Cas shown in Figure 1A. 
[0027] In Figure 1 A, a portion of the mouse 100 com- 
prises a rotatable physical surface contact device, here 
illustrated as a track ball detector 110, further compris- 
ing a track ball 112. The track ball surface 114 of the 
track ball 112 frictionally engages a surface area 130 
over which the mouse 100 travels, preferably in atrans- 
lational or 'X-Y 1 type movement. 
[0028] The spacing between the mouse housing 1 06 
and the surface 1 1 4 of th e track ball 1 1 2 allows the track 
ball to freely contact and engage the surface 1 30. The 
track ball detector 1 1 0 may preferably be of the type pre- 
viously described and utilized in conventional computer 
mice as incremental encoders, e.g., similar to the incre- 
mental encoder 26 shown in Figures 9-10. In Figure 1A, 
the track ball detector 110 provides an output indicating 
the relative displacement of the mouse along the sur- 
face 1 30. Although not shown, a computer simi lar to the 
CPU shown in Figure 7A is preferably utilized, along with 
software, so that the location of the mouse 100 may be 
indicated as a cursor on a computer display device. 
[0029] In order to provide a stable support during a 
translational movement of the mouse 100, a plurality of 
support feet 1 58 are attached to the mouse housing 1 06 
so as to support the housing above the surface 1 30. 
[0030] Another portion of the mouse housing 1 06 is 
comprised of switches, herein illustrated as mouse but- 
tons 1 40. Mouse buttons 1 40 may comprise any number 
of buttons from one to as many as desired by the user 
in the practice of this invention. In this arrangement, as 



shown in Figure 1B, three buttons are preferable. The 
mouse buttons 140 comprise a right mouse button 142, 
a center mouse button 1 44, and a left mouse button 1 46. 
The mouse buttons are preferably utilized by the user 

s as input devices. Although not preferable in this ar- 
rangement, it is also possible to practice this invention 
without utilizing or providing any mouse buttons. 
[0031] In this arrangement, as shown in Figure 1 A, the 
mouse 100 is preferably positioned, e.g., by the user, 

10 over a portion of the surface area 1 30 that comprises 
an area of interest. This area of interest may also be 
referred to as an area to be sensed or, namely, is an 
area to be imaged 1 32. 

[0032] Preferably, the area to be imaged 1 32 reflects 
a first portion of light traveling from the area to be imaged 
132 to a reflector surface 157 so as to comprise light 
path 124. Light traveling along light path 124 is then in- 
cident upon reflector surface 157 that comprises a por- 
tion of a reflector 156 that is suitably positioned in the 

20 interior of the mouse housing 106. A portion of the light 
incident upon reflector surface 1 57 is reflected and trav- 
els from the reflector surface 1 57 to the sensor unit 1 20 
along a light path 125. The light that travels along light 
path 125 is delivered to the imaging unit, namely, a sen- 

25 sor array 122 contained in the sensor unit 120. 

[0033] However, another portion of the light that trav- 
els along light path 124 is not reflected by reflector sur- 
face 157, but instead passes through reflector 156 so 
as to travel along light path 1 26, and thus also passes 

30 through view port area 1 52 so as to exit the mouse hous- 
ing 106. 

[0034] A sensor port area 1 54 is provided on the side 
of the mouse housing 1 06 that is adjacent to the area to 
be imaged 132, so that light is admitted into the housing 
3S 106 along light path 124, Light may be provided from a 
plurality of sources, of which only two sources are 
shown in Figure 1A. 

[0035] For example, one source may comprise ambi- 
ent light that may be admitted through an optional illu- 

40 mination window 1 70, so as to provide sufficient light to 
be reflected by the area to be imaged 1 32. Alternatively, 
one or a plurality of light emitting diodes ("LED") may 
form light source 172 to provide light along light path 
1 28 to the area to be imaged 1 32. Of course, if desired, 

45 both these light sources may be used together. 

[0036] Depending upon the size and shape of the 
mouse housing 106 desired, it is also possible to prac- 
tice this invention without either the LED light source 1 72 
nor the relatively large illumination window 170. Instead, 

so ambient light, for example, as may be found in an office, 
home or school environment may be adequate, depend- 
ing upon, e.g., the type and quality of sensor array 122 
that is chosen to be utilized, and which operating mode 
or modes are desired, in the practice of this invention. 

55 [0037] Upon receiving the reflected light, the sensor 
array 1 22 provides data to be output by the sensor unit 
120 along mouse bus 104. In Figures 1A-C, the "X-Y 1 
location of the mouse is determined by the track ball de- 
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lector 1 1 0, and its representative output signal is also 
transmitted along the mouse bus 104. Also, the input 
signals from the mouse buttons 140 are transmitted 
along the mouse bus 104. Mouse bus 104 may be uti- 
lized both to communicate among the above -de scribed 
various portions of the mouse 100, and may also be uti- 
lized to transmit signals and data between the mouse 
and an external device by utilizing mouse tail bus 102. 
[0038] Of course, while Figures 1-3 are shown with a 
mouse tail bus 102 comprised of a "hard 1 connection, 
e.g., wires or fiber optic cables, the mouse tail bus 102 
may also comprise a link provided through an infra-red 
connection, or may comprise any other wireless, e.g., 
light, radio frequency, or acoustic link transmission tech- 
niques. 

[0039] The sensor array 1 22 may comprise, lor exam- 
ple, a charged coupled device ("CCD"), or comprise a 
Complimentary Metal Oxide Silicon ("CMOS") imager 
array. The sensor array 122 will be described in more 
detail in accordance with Figure 6. 
[0040] The sensor unit 120 is capable of imaging al- 
most anything contained on the area to be imaged 132. 
For example, Figures 1B and 1C illustrate a machine 
decodable symbol 160 that may be detected by the im- 
ager mouse 100. A preferable example of symbol 160 
is a bar code. 

[0041] As shown in Figures 1A-C, light is reflected 
from the symbol 160 and travels to the reflective surface 
157 along light path 124 and then along light path 125 
to the sensor array 122. Because of the orientation of 
light path 125, the sensor array 122 is considered to be 
pointing along a horizontal axis. In this arrangement, the 
vertical axis is comprised of light path 124 and light path 
1 26. The semi-reflective surface 1 57 of the reflector 1 56 
enables the sensor array and its focusing optics to be 
positioned off of, or removed from, the vertical optical 
axis with respect to the area to be imaged 1 32. 
[0042] There is an advantage of pointing along a hor- 
izontal axis as represented by light path 1 25. The ad- 
vantage is that the field of view, or optical focus area, of 
the sensor array 122, need not be as "short" as may be 
required if the sensor array 122 were located more op- 
tically close to the area to be imaged 1 32, so as to re- 
ceive light directly from the vertical axis. 
[0043] As shown in the preferred arrangement of Fig- 
ures 1 A-C, the user is provided with a see-through op- 
tical view port area 152 so that the user may view the 
area to be imaged 1 32 while the user is positioning the 
mouse. Alternatively, instead of the view port area 152, 
a liquid crystal display ("LCD") device, not shown, or oth- 
er similar display device may comprise a portion of 
mouse 100. The alternate displays may be located in 
the same location as the mouse housing 1 06 that the 
viewport area 152 occupies, if desired. In another alter- 
native arrangement, both the view port area 1 52 optical 
window and the LCD display device may be provided. 
Of course, these viewing arrangements may each be 
located in areas other than that shown in Figures 1 A-C 



in the practice of this invention. Also, the LCD display 
may be replaced by any other type of electronic display 
device. 

[0044] Yet another alternative arrangement is to uti- 
5 lize the mouse internal bus 104 with either the mouse 
tail bus 102, or with any of the alternative external com- 
munication devices and methods previously described, 
so as to provide for a displayed image of the area to be 
imaged 1 32 on an external or remote display device, for 
example, a computer CFTT. The image of the area to be 
imaged 1 32 is preferably displayed in "real time," as the 
mouse 1 00 is being moved or positioned by the user. 
This embodiment may be used with or without the view 
port area 152. In another alternative arrangement, the 
view port area 1 52 may be imp lemented as a magnifying 
lens to better enable the user to view the area to be im- 
aged 132. 

[0045] However, in the arrangements illustrated in 
Figures 1A-C, the mechanical track ball detector 110 
has the same problems with accumulation of dirt and 
dust that conventional mice have. Thus, the arrange- 
ments as shown in Figures 2A-C and 3A-D are prefer- 
able to the arrangements described in Figures 1A-1C. 
[0046] Figures 2A-C illustrate an arrangement of the 
mouse device 100, wherein Figure 2A illustrates aside 
view of the mouse 100, Figure 2B illustrates a top view 
of the mouse 100 of Figure 2A, and Figure 2C illustrates 
a top cross-sectional view of the mouse 100 shown in 
Figure 2A, the section being taken along the line 2C-2C 
in Figure 2A. 

[0047] Unlike the mouse 100 device illustrated in Fig- 
ures 1 A-C, the mice 1 00 of Figures 2A-C and 3A-D elim- 
inate the track ball detector 100 and the track ball 112 
of the mouse 100 illustrated in Figures 1A-C. Thus, the 
translational, or X-Y, movement of the mouse is detect- 
ed solely by the sensor unit 120, by utilizing the sensor 
array 122. For example, in Figure 2A, sensor array 122 
receives light reflected from the area to be imaged 1 32, 
along light path 224. The light reflected from the area to 
be imaged 1 32 passes through a sensor port area 154 
that allows light to be admitted to the mouse housing 
106 and continue on to the sensor array 122. 
[0048] The light source may, for example, be provided 
by ambient light, wherein the ambient light may be gen- 
erally available in the environment that the mouse is uti- 
lized in. Alternatively, ambient light may be emitted 
through an optional illumination window 170, as is the 
case for mouse 100 described in Figures 1A-1C. 
[0049] Another alternative is to provide an LED illumi- 
nation 172. However, unlike the mouse 100 illustrated 
in Figures 1A-C. Figure 2A illustrates an LED illumina- 
tion 172 that is provided in a plurality of regions, here 
shown as LEDs at the opposite ends of the sensor array 
122. Although not shown, the LED illumination may also 
be provided as a plurality of LEDs, each relatively small 
in relation to the size of the sensor unit 120, such that 
the plurality of LEDs are distributed around and outside 
the field of view of the sensor array 1 22. For example, 
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the LEDs may be distributed so as to roughly form a cir- 
cle or a rectangle at a similar relative height, above the 
surface 130, as the LED illumination 172 elements are 
shown in Figure 2A. 

[0050] Also simi lar to the mouse 1 00 in Figu res 1 A-C, 
the mouse 100 of Figure 2 is maintained a preferred dis- 
tance above the surface 1 30 by utilization of support feet 
158 on the front, or sensor, end of the mouse 1 00. In a 
preferred alternate arrangement, Figures 2A-C illustrate 
a support bar 159 that replaces the support feet 158 of, 
for example, mouse 100 of Figures 1 A-C, so that sup- 
port is generally continuous along the backside, or 
mouse tail 102 portion, of the mouse 100 of Figure 2B. 
Alternatively, this support bar 159 configuration may be 
utilize along the left and right sides, or on the front por- 
tion of mouse 100. For example, a second support bar, 
not shown, that is similar to support bar 159, may re- 
place the two front support feet 158 that are currently 
illustrated in Figure 2B. 

[0051] Figure 2A illustrates another alternate ar- 
rangement of mouse 100. For example, instead of im- 
aging light reflected from the surface along light path 
224, the light sensor portion of the sensor unit 120 may 
be instead located along a front portion of the sensor 
un it, so as to comprise alternate sensor array 1 22F. Sen- 
sor array 122F preferably senses light along light path 
324 that is reflected generally from in front of the mouse. 
Sensor array 122F may utilize illumination window 170 
to allow light along light path 324 to enter the mouse 
housing 106, so as to reach sensor array 122F. This al- 
ternate arrangement allows for the user to view the al- 
ternate area to be imaged 1 32F so as to enable the user 
to easily move and position the imaging mouse 100. 
[0052] Alternately, and preferably when utilizing light 
path 224 to reach sensor array 122 of Figu re 2A, at least 
one of a real-time display on a computer display device, 
or an LED display device located, for example, on an 
upper portion of the mouse housing 106, is provided to 
give the user a real-time view of the area to be imaged 
132. Implementation of any of these view ports, LCD 
view screens, or external display devices, may be sim- 
ilar to that as described previously for Figures 1 A-C. 
[0053] Also as in Figure 1 A-C. the mouse 100 of Fig- 
ures 2A-C comprises preferably 3 mouse buttons 140, 
further comprising the left, center, and right mouse but- 
tons 146, 144 and 142, respectively. Also, mouse tail 
bus 102 may be utilized to send or receive X-Y location 
and/or video data between an external device such as 
a CPU and the mouse unit. 

[0054] Of course, for any of the devices described in 
Figures 1 -3, the mice 100 may alternately be, for exam- 
ple, battery-powered. The mice 100 may also utilize, for 
example, the wireless connections previously de- 
scribed, if desired, between the mouse 100 and an ex- 
ternal device such as a CPU. Alternately, the mouse 
may provide an LED display device itself, as previously 
described, and also store at least a portion of the image 
data and/or X-Y location data for later transmission to, 



for example, a CPU. 

[0055] Similar to the mouse 100 in Figures IA-C, the 
mouse 100 shown in Figures 2A- 2C, comprising the 
sensor unit 120, utilizes sensor array 122 to sense the 
5 area to be imaged 1 32, and thereby delerm ines relative 
movement of the mouse 100 with respect to the surface 
1 30. In this preferred arrangement, the mouse 100 may 
utilize a variety of methods, e.g., apply digital process- 
ing techniques to the sensed data, to determine this rel- 
ative movement. 

[0056] For example, the field of view of sensor array 
122 may be more closely focused on the area to be im- 
aged 1 32 so as to detect material fibers 1 66 (Figu re 2C), 
such as wood fibers of a desk, or paper fibers contained 
in a conventional desk mat or sheet of paper. In another 
alternative arrangement, the sensor array 122 detects 
an image comprised of , for example, random or pseudo- 
random geometric designs 162 or other fanciful artistic 
designs. Of course, a completely non-random and re- 
petitive pattern may be utilized as well. In yet another 
alternative arrangement the sensor array 122 detects 
an image comprised of, for example, some repetitive 
pattern or distinctively recognizable figure, such as stick 
people artwork 164, or a reproduction of the image of 
Leonardo da Vinci's Mona Lisa, that is not shown. 
[0057] Essentially, the mouse 100 functions for the 
user in the same way a conventional mouse does, for 
example, providing a cursor on a computer display. 
Quite different, however, is that the mouse 100 senses 
any motion induced by the operator on the mouse 100 
through the sensor array 1 22. The sensor array 1 22 and 
downstream software to be described below, detects the 
relative movement of artifacts across its field of view in 
any arbitrary direction within the focal length setting of 
the mouse. 

[0058] Thus, any of these images may be utilized by 
the mouse 1 00, because the sensor array 1 22 detects 
X-Y relative motion from changes in a plurality of edges 
of the image. For example, the plurality of edges may 
be sensed as a result of relative contrasts preferably oc- 
curring throughout the area to be imaged 1 32. This pro- 
vides the user with great discretion as to what surface 
and what images to use, and in fact requires no conven- 
tional mouse pad whatsoever. 

[0059] Of course, a 'special" mouse pad may be pro- 
vided for the imager mouse to use with a specific design, 
such as a grid pattern upon it. However, this should not 
be confused with the previously described conventional 
special reference grid pad, that provides contrast with 
two wavelengths of light, one for each axis. 
[0060] More importantly, the instant imager mouse 
100 does not have the problems encountered with the 
previously described conventional special pads utilized 
with the IR and visible LED utilized for the first and sec- 
ond axes, respectively. Instead, the instant invention op- 
erates as the user may intuitively expect. In other words, 
even with using a special pad, the imager mouse 100 
would not be required to orient the cursor movement to 
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the x and y grids on the pad. Even if the pad were rotat- 
ed, the X-Y coordinates would not necessarily change. 
Thus, the user may experience an intuitive movement 
of the cursor. 

[0061] This preferred intuitive result occurs because, 
instead of relying on the reference grids as the conven- 
tional optical mouse does, the instant imager mouse 1 00 
utilizes the sensor array 122 in combination with digital 
processing of pixel array data supplied by the sensor 
array 122, to detect the direction and amount of move- 
ment relative to the surface. Thus, in the instant inven- 
tion, while there is no requirement for a special pad, 
such a special pad may be used, just as almost any sur- 
face may be used. 

[0062] Further, by eliminating the track ball 112 and 
track ball detector 114, the problem of accumulation of 
dust and dirt in those mechanical assemblies is elimi- 
nated. More importantly, by utilizing the sensor array 
122 as described herein, a more responsive mouse may 
be realizable. 

[0063] For example, with a mechanical tracking de- 
vice, the mouse may require some significant move- 
ment, particularly when it has become dirty or dusty, in 
order to cause the cursor to respond on the screen. 
However, in the case of the imager mouse 1 00 of Fig- 
ures 2-3, the user may set the mouse 100 so that even 
very fine or very small movements of the mouse may 
result in significant movement of the cursor. Of course, 
the amount of cursor movement relative to the imager 
mouse 100 is preferably variable so that the user may 
specify that, for example, a great deal of movement of 
mouse 1 00 may be requ ired in order to cause the cu rsor 
to move on the display. Again, this relative movement 
of the cursor to the movement of the imager mouse 1 00 
is something the user may change or define according 
to the user's preferences, without the inherent limita- 
tions of a conventional mechanical track-ball -type 
mouse. 

[0064] The mouse 1 00 may focus on fine details, for 
example, such as paper fibers. This may require a sen- 
sor array 1 22 with a relatively short field of view, or with 
a considerable number of pixels, or both, depending, e. 
g., upon the optics utilized. Alternatively, the field of view 
of the sensor array 122 may be enlarged, or widened, 
to allow the imager to detect only, for example, relatively 
larger features imprinted on the surface 1 30. However, 
in each case, the imager mouse 100 may function es- 
sentially the same, in that it may detect edges of the 
image and translate a relative movement of these edg- 
es, e.g., by signal processing, into X-Y coordinates for 
display as a cursor on a display device. 
[0065] As shown in Figure 2A, when the mouse 100 
operates to sense the area to be imaged 132F that is 
essentially in front of the mouse, the sensor unit 120 is 
preferably configured with an alternate reflector, either 
fixed or adjustable, that may be similar to that shown in 
Figure 1 A. The alternate reflector, not shown, operates 
to ultimately direct the light that initially travels along 



light path 324 into a horizontal path similar to the light 
path 125 as shown in Figure I A. This alternate forward 
sensing operation preferably comprises a mouse 100 
that is considered to be pointing along a horizontal axis, 
s as previously described. Of course, this alternate for- 
ward sensing arrangement of an area to be imaged 
1 32F may be implemented on any of the mice 100 illus- 
trated in Figures 1-3. Also, the sensor array 122F may 
alternately be either fixed or adjustably mounted at an 
10 angle to best sense the area to be imaged 1 32F so that 
the alternate reflector may be omitted. 
[0066] However, in the mouse 100 illustrated in Fig- 
ures 2A-C, the sensory unit 120 is preferably oriented 
so as to be pointing along a vertical axis, i.e., the vertical 
axis aligned with the light path 224. In this arrangement, 
it is somewhat more difficult to provide the user an op- 
tical 'see-through 1 window technique of viewing the rel- 
ative alignment of the sensor array 1 22 over the area to 
be imaged 1 32. Instead, the sensor array 120 preferably 
20 communicates with a computer, not shown, to display 
an image that may be utilized to aid in the fine position- 
ing of the mouse 1 00. Alternately, the sensor array 1 20 
communicates with a display, not shown, that is on 
board the mouse 1 00 to display an image for fine posi- 
es tioning of the mouse 100. 

[0067] As previously described, illumination is prefer- 
able for operating the instant invention as a mouse to 
provide a cursor on a display particularly when practic- 
ing the invention as illustrated in Figures 2-3. The pre- 
30 f erred embodiment of a light source other than ambient 
light is to utilize an LED, preferably white. A second pref- 
erable LED color is red. However, a white LED will allow 
the mouse 100 to function over a broader range of 
colors. 

3S [0068] In any of the arrangements illustrated in Fig- 
ures 1 -3, the light source is preferably an array of LEDs, 
so as to provide more uniform lighting of the area to be 
imaged 1 32. The LED array is also preferably provided 
with a technique for switching the LED array on and off, 

40 the switching being preferably under the control of the 
sensor unit 120. 

[0069] Alternatively, for example, clear or diffuse plas- 
tic parts may also be utilized to provide more illumina- 
tion, allowing for a minimum number of LED light sourc- 
es es. Another alternative light source is to utilize laser 
light. Also, alternatively, a pulsed laser or strobed LED 
type arrangement may be utilized. 
[0070] However, it will be understood that the sensor 
array 122 does not require collimated light. Also, while 
so the imager preferably utilizes a focusing lens, the sensor 
array 122 does not require a collimating lens in order to 
sense the area to be imaged 132. This is a improvement 
over conventional optical mice. 
[0071] One concern in practicing the invention arises 
55 when the mouse 1 00 of the embodiments of Figures 1 -3 
is intended to be utilized intuitively as a conventional 
mouse, but the mouse 100 is moved primarily along a 
Z axis, in this case parallel with the light path 224 as 
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shown in Figures 2A-3C. Because the sensor unit 12 
will continue to image and sense relative movement, 
even when the mouse 100 is lifted from the surface 1 30 
and substantially removed from the area to be imaged 
132, at least one of the following techniques may be 
practiced, so as to have the mouse 100 appear to func- 
tion in an intuitive manner. 

[0072] One technique is to provide for the mouse 1 00 
to stop issuing X-Y position update commands whenev- 
er the mouse is lifted from the surface 1 30. This com- 
mand stoppage may be accomplished, for example, by 
utilizing the above described illumination LEDs in a 
strobe light mode of operation. The strobe light mode 
allows the sensor array 1 22 to be exposed to at least 
two different light levels, for example, illuminated and 
ambient. The illuminated and ambient conditions corre- 
spond to the LED being on and off, respectively. The 
imaging sensor unit 120 located within the mouse hous- 
ing 106 strobes the LED on and off, and utilizes the sen- 
sor array 122 to measure the overall light level detected 
from the area to be imaged 132. The sensor unit 120 
compares the relative light levels observed under vari- 
ous conditions and utilizes the results to detect when 
the mouse is being lifted from the operating su rface 1 30. 
[0073] One example of the need for this shutdown of 
X-Y position update commands is when the mouse user 
wishes to move a cursor in a direction across a display 
that is representational ly further than that allowed by the 
physical desktop space available to the user for moving 
the mouse 100. If the X-Y position update commands 
are suppressed or stopped when the mouse 100 is not 
in contact with surface 130, then the mouse 100 will 
function intuitively when the user, e.g., lifts, repositions, 
and then again engages the surface 1 30 with the mouse 
100 in order to complete a movement of the cursor to 
the desired position on the display. 
[0074] In practicing the strobe light technique, some 
alternate methods and conditions that indicate to the 
sensor un it 1 20 that the mouse 1 00 is in contact with the 
surface 130 may include the following. 
[0075] For example, when the sensor unit 1 20 detects 
changes in overall light level that are synchronized with 
the LED switching, or turning On and Off, the sensor unit 
120 may determine that the mouse 100 is receiving light 
that is reflected back into the sensor array 1 22 from the 
mouse surface that has originated from the LEDs. This 
condition may be interpreted as a "mouse down" condi- 
tion or mode, i.e., the mouse 100 is in contact with the 
surface 130. 

[0076] Alternatively, a condition that indicates that the 
mouse 100 is not in contact with the surface 130, i.e., a 
"mouse lifted" condition or mode, may include the fol- 
lowing. 

[0077] When the overall light level received by the 
sensor array 122 does not change significantly in re- 
sponse to the strobing of the LEDs, or e.g., does not 
appear to be sufficiently synchronized with the LEDs, 
then the sensor unit 1 20 may determine that there is not 



a significant amount of light being reflected back into the 
imager or LED array. If the sensor 5 unit 120 determines 
that one of these conditions has occurred, the sensor 
unit 120 will shut down the mouse cursor signal until 

s such time as the sensor unit 120 determines that the 
light is sufficiently synchronized, or is sufficiently chang- 
ing, so as to indicate the mouse is in a reasonable prox- 
imity to a surface 1 30, so as to start or resume providing 
X-Y position update commands. 

10 [0078] When the mouse 1 00 is removed from the sur- 
face 130, the observed light level will depend much 
more on the ambient light level than on the amount of 
light emitted from the LED array, as long as the mouse 
is far enough from a diffuse reflecting surface, or is not 

is optically aligned with a mirror-like surface. However, this 
method may have some operational concerns when 
used on an exceptionally shiny or reflective surf ace such 
as glass or metal. 

[0079] In order to strobe the LEDs, for example, a sim- 
20 pie algorithm may be utilized that may provide two light 
levels, the first comprising all LEDs switched "On" and 
the second level comprising all LEDs switched 'Off. ' 
Alternatively, mouse 100 may generate multiple levels 
of light by selectively activating groups of LEDs. The 
25 mouse 100 may then determine, e.g. by utilizing soft- 
ware, whethertoturnallofthe LEDs off, or only a portion 
of them, depending upon the primary use anticipated, 
or designated, by the user. 

[0080] Another alternative for stopping the issuing of 

30 x-Y position update commands by the mouse 1 00 com- 
prises utilizing a processor, to be described in detail later 
with Figure 6. The processor preferably comprises a 
portion of the sensor unit 120, but may also be hosted, 
e.g., in a CPU, that is not shown. The processor may 

3S essentially interprets, or "sees," the area to be imaged 
132 generally going out of focus when the mouse 100 
is lifted from the surface 1 30. A combination of software 
and/or hardware may be utilized with the processor, or 
sensor unit 120, that would utilize a threshold limit of 

40 focus, or relative distance from the area to be imaged 
1 32. as desired by the user. For example, upon reaching 
or exceeding the threshold limit of focus, the processor, 
or sensor unit 1 20, would stop the issuing of X-Y position 
update commands by the mouse 100, even though the 

45 mouse may continue to sense or scan, utilizing the sen- 
sor array 122. However, the issuing of update com- 
mands would not continue until the processor, or sensor 
unit 120, determined that the mouse was again within a 
suitable distance of an object or surface so as to be with- 

so h the threshold limit of focus. 

[0081 ] Yet anothe r technique for stopping the X-Y po- 
sition update commands by the mouse 100 comprises 
a mechanical sensor, e.g., a microswitch, not shown, 
that is mounted to the portion of the mouse housing 1 06 

55 that may be adjacent to the surface 130 illustrated in 
Figures 2-3. The microswitch, or a plurality of micros- 
witches, may be integrated with the support feet 158 
and/or the support bar 159. By utilizing a microswitch 
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technique, any concerns of a shiny or reflective desktop 
of metal or glass may be alleviated. 
[0082] Yet other alternate arrangements, as de- 
scribed below, that may desire to control the issuing of 
X-Y position update commands include portable 
processing devices such as, for example, a portable or 
pen based computer, or a PDA device. In these portable 
devices, a thumb, finger, or hand operated microswitch- 
like" device may be implemented to control the issuing 
of X-Y position update commands. In these portable de- 
vices, the imager may be oriented, for example, so as 
to point "up 1 at the user. In one example, a finger may 
dep ress a microswitch , or a pressure pad function ing as 
a microswitch-like device, so as to enable the output of 
update commands. Other various similar arrangements 
may also be implemented. 

[0083] Other arrangements for controlling the issuing 
of X-Y position update commands in these portable 
processing devices may utilize various arrangements of 
the LED illumination for Z-axis sensing described here- 
in. Also, for example, another technique may utilize IR 
LEDs in the portable data processor configurations in 
order to check for the close presence of an object, e.g., 
a finger or a bar code. One reason for utilizing the IR 
LEDs in either of the portable data processors, or even 
in the previously described desktop mouse configura- 
tions, is to reduce subjecting the user to a potentially 
annoying source of flicker. Another advantage of utiliz- 
ing the I R LEDs may be realized because silicon imag- 
ers, including some of the examples described herein, 
may be relatively quite sensitive in the IR portion of the 
spectrum. 

[0064] Also, a relatively low duty cycle may be utilized 
with the LEDs in order to minimize battery drain on a 
portable device. These alternate IR arrangements may 
also be understood to be similar to a "wake-up" type cir- 
cuit or device with a pulsed IR emitter. 
[0065] Of course, for any of these alternate arrange- 
ments, it is understood that only a subset of the imaging 
array may need to be processed for a light level re- 
sponse to the LED flickering or pulsed IR emitter. Fur- 
ther, this reduced image size may result in a reduced 
data load, that may then result in faster sample rates. 
Alternately, this reduced image size may result in less 
power consumed in sensing, or 'looking,' for an object 
to move relative to, or approach, the imager, e.g., a fin- 
ger or a bar code. This less power consumed arrange- 
ment may be particularly desirable for a battery-pow- 
ered and/or handheld device. 

[0066] Further, another alternative arrangement pro- 
vides for combining the strobe light technique with the 
microswitch technique to allow the sensor unit to have 
inputs from both techniques, in order to make decisions 
as to when to start or stop the X-Y position update com- 
mands. 

[0087] Yet another alternative is to provide a sensing 
material or sensor pad, e.g., as a portion of at least one 
support foot 158 and/or support bar 159, where the 



sensing pad is similar to a conventional touch pad on a 
laptop computer. Whenever physical pressure is applied 
to the pad, a signal may be provided to the sensory unit 
1 20 that indicates that the mouse is in the 'mouse down" 
5 mode. Also, when the pad does not sense adequate 
pressure, the signal may be changed so that the sensor 
unit may put the mouse intothe 'mouse lifted" mode and 
thus stop the provision of X-Y position update com- 
mands. Of course, any or all of the above methods for 
Z axis movement detection, e.g., the microswitch, pres- 
sure pad, processor interpretation, laser pulse, or LED 
strobing, may be combined as desired to allow the 
mouse 100 to intuitively operate. 
[0068] It will be understood to one skilled in the art 
that any of the above sensors, e.g., micros witches or 
sensor pads, are not actually determining any X-Y 
movement. They are only involved in the determination 
of a Z movement. Thus, the mice 100 of Figures 2-3 do 
not need to rely upon any mechanical parts to determ ine 
a relative X-Y movement. 

[0069] As shown in Figure 2A-C, the mouse internal 
bus 104 is preferably utilized to provide the communi- 
cation between, for example, the strobe LEDs 172 and 
the sensor unit 120, as well as to provide a communi- 
cation link to the external display device via the mouse 
tail bus 102. Also, any signals or communication re- 
quired between any sensors, e.g., micros witches in the 
support feet 158 or support bar 159, may also be pro- 
vided with connections to the mouse internal bus 1 04, 
although these additional connections are not shown in 
Figures 2A-C. 

[0090] In summary, the mouse 100 and any of its al- 
ternatives, as illustrated in Figures 2A-C, comprise an 
imaging mouse that may be utilized for cursor position- 
ing on a computerized display. 

[0091] Figures 3A-C illustrate a code reading and 
mouse device 100 resulting from the practice of another 
embodiment of this invention. Figure 3A illustrates a 
side view of the mouse 100, and Figure 3B illustrates a 
top view of the mouse 100 illustrated in Figure 3A. Fig- 
ure 3C illustrate a cross-sectional top view of the mouse 
100, wherein the section is taken along the line 3C-3C 
in Figure 3A. 

[0092] Figures 3A-C illustrates a computer mouse 
100 that preferably incorporates essentially all of the 
functions of the imaging mouse 100 of Figures 2A-C, 
and, in addition, the sensor array recognition of preprint- 
ed symbols as described in Figures IA-C. The mouse 
100 of Figures 3A-C preferably comprises a single sen- 
sor unit 1 20 util izing a single sensor array 1 22 to perform 
the imaging that is performed for both the mouse trans- 
lational movement determination and the symbol imag- 
ing that includes bar code scanning. 
[0093] As in Figures 2A-C, the sensor unit 122 of Fig- 
ures 3A-C is essentially comprised of the same compo- 
nents, and preferably utilizes a technique of pointing 
along a vertical axis as previously described with Fig- 
ures 2A-C. Of course, the alternative support bar 159 
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illustrated in Figures 2A-C may be utilized instead of the 
support feet 158 of Figures 3A-C, if desired. 
[0094] In another arrangement, a skirt 407, as shown 
in Figures 3A-B, is provided around the perimeter of the 
housing base portion 406 of mouse 100. For example, 
the skirl may comprise a curtain-like wall, protrusion, or 
flap. The skirt 407 may be implemented around the pe- 
rimeter of any of the mice 100 of Figures 1-3. The skirt 
may be comprised of a relatively firm skirt portion 507, 
and a relatively flexible skirt portion 607. The relatively 
firm skirt portion 507 may be comprised, e.g., of plastic 
similar in firmness to a plastic that may be utilized in the 
construction of the housing 106. The relatively flexible 
skirt portion 607 may be comprised, e.g. of flexible rub- 
ber, a relatively flexible plastic, a flexible sponge-like 
material, or cloth. 

[0095] The relatively firm skirt portion 507 is closest, 
and attached, to the housing base portion 406. The flex- 
ible skirt portion 607 is attached to the firm skirl portion 
507. The flexible skirt portion 607 preferably continues 
a physical extension to the surface 130, that is initiated 
from the housing base portion 406 by the firm skirt por- 
tion 507. 

[0096] The firm and flexible skirt portions cooperate 
to extend from the housing base portion 406 to the sur- 
face 1 30, in order to provide a more controlled light en- 
vironment for the sensor unit 122. A more controlled light 
environment is provided because the skirt 407 prefera- 
bly blocks either a portion or substantially most, if not 
all, of the ambient light from entering the area to be im- 
aged 132. This allows, e.g., the LEDs to provide a more 
controlled lighting condition of the area to be imaged 
132. 

[0097] As shown in Figure 3C, the sensor unit 1 20 uti- 
lizes the sensor array 1 22 to sense an area to be imaged 
1 32 that may comprise any or all of the following. 
[0098] The area to be imaged 1 32 may comprise, for 
example, the previously described random or pseudo- 
random geometric configurations or fanciful artistic de- 
signs 162, the artwork 164, or the material fibers 166. 
[0099] The area to be imaged 1 32 also may comprise 
machine decodable symbols, such as a "one-dimen- 
sional' symbol 160, e.g., a bar code, or a "two-dimen- 
sional 1 symbol 161. The two-dimensional symbol 161 , 
for example, may be a pre-printed matrix for digital sens- 
ing and interpretation, or any other machine decodable 
symbol. Also, the area to be imaged 132 may comprise 
graphics and text 163. The sensor unit 120 and/or a 
computer device that is not shown, may optionally per- 
form optical character recogn rtion ("OCR") on the graph- 
ic or text image 168. 

[0100] In imaging the machine decodable symbols, 
another arrangement of the mouse 100 includes the 
sensor unit 120 capable of "self-learning. " The sensor 
unit 120 retains its experience about the machine deco- 
dable symbols, or alternately has, e.g., a library of all, 
or at least the most likely, symbols and their formats so 
that the sensor unit 120 may assist the user by making 



an "educated guess 1 as to which symbol it is likely sens- 
ing. This helps improve performance overtime and also 
may be applied to interpretation of X-Y movement 
across the surface 130. 

s [0101] For example, self -learning of the image pat- 
tern, or design, that serves as a regular surface 1 30 that 
the mouse 100 is moved upon, may be learned and re- 
tained, along with other repetitively used surfaces. This 
self -learning may assist the sensor unit 120, e.g., in the 

10 determination or the comparison of the edges detected 
with its memory of edges previously encountered for 
that surface, so that the sensor unit 120 may more ac- 
curately, or easily, determine its X-Y location and rela- 
tive movement. 

*5 [0102] The imaging mice 100, as illustrated in each of 
the Figures 1A-3C, preferably utilize a single non-ad- 
justable lens. For example, the lens may be formed ad- 
jacent to and upon the sensor array 122 itself, or may 
be located separate thereto. However, an adjustable 

£0 lens arrangement may be utilized as is illustrated in Fig- 
ures 3D and 5A-B. 

[0103] Figure 3D illustrates yet another embodiment 
of the computer imaging mouse 100D resulting from the 
practice of this invention. As in Figures 3A-C, the imag- 
es ing mouse 1 00D may use its imaging capabil ities to pro- 
vide X-Y relative movement, and capture printed indicia 
such as machine decodable symbols. 
[0104] In Figure 3D, imaging mouse 100D comprises 
a mouse housing 106D that contains a sensor unit 120. 
30 Except as follows, the mouse 100D is generally similar 
to the mice 100 as described in Figures 1A-3C. 
[0105] In Figure 3D, the mouse housing 106D com- 
prises a first mouse input device 242, a second mouse 
input device 244, a third mouse input 246, and a fourth 
3S mouse input device 248. The mouse input devices 242, 
244, 246, and 248 may comprise electrical switches, or 
mouse buttons, or alternatively may comprise, e.g., 
pressure sensitive areas. 

[0106] The mouse input devices 242, 244, 246, and 
40 248 transmit a signal to the sensor Unit 120 along the 
mouse bus 104. If transmission external to the mouse 
100D is desired, the mouse signals may be transmitted, 
as previously described in Figures 1 A-3C, along mouse 
tail bus 102, or alternatively by wireless connection. 
45 [0107] The sensor unit 120 utilizes a technique, as 
previously described, of pointing along a horizontal axis, 
or alternatively, a modified pointing along a horizontal 
axis. Thus, the sensor array may receive light directly 
from the area to be imaged that is located essentially 
so below the viewport area 352 of Figure 3D. 

[0108] The area to be imaged, that is generally locat- 
ed underneath the view-port area 352, is also generally 
defined by a left front arm 308L and a right front arm 
308R, wherein both arms form a portion of mouse hous- 
es hg 106D. These arms serve as a visual guide to the 
user to position the mouse appropriately over a code or 
symbol for imaging thereof. As such, these arm guides 
serve a similar function as the view port area 1 52 of Fig- 



12 



21 



EP 0 957 448 A2 



22 



Lire 1A. 

[0109] As in the mice 106 of Figures 1 and 3, the 
mouse 100D of Figure 3D is capable of imaging cap- 
tured printed indicia such as machine decodable sym- 
bols, for example, a one-dimensional bar code 160 and 
a two-dimensional symbol 161, as shown in Figure 3D. 
[0110] Unlike the mice 100 of Figures 1A-3C, mouse 
100D as shown in Figure 3D, may utilize four mouse 
burtons. This is to allow an assignment of one of the 
buttons as a mouse trigger control. One example of this 
mouse trigger control is to utilize it as a " mode control 
button." This mode control button may enable a user to 
switch the imaging mouse 100D among its various 
modes of operation . Any of the four mouse inp ut devices 
242, 244, 246, or24B may be designated by the user as 
a default mode control button. It is preferable that the 
user be able to specify which mouse input device the 
user desires to use, and may be based upon, e.g., the 
users convenience, or ergonomic satisfaction. 
[0111] For example, depressing the designated trig- 
ger button once may switch mouse 1 0OD from a "mouse 
mode" to a "code reader mode. 1 Depressing the desig- 
nated trigger button again may switch the mouse 1 0OD 
from the "code reader mode" to a 'scanner mode," and 
depressing it yet again may bring the mode back to the 
initial "mouse mode," Another example of a mode is a 
'video camera mode." 

[0112] The "mouse made," the "code reader mode," 
"and the "scanner mode," operate similar to those tech- 
niques described for Figures 1A-3C above. For exam- 
ple, in mouse mode, the mouse 106D operates to pro- 
vide an X-Y relative movement of the mouse over a sur- 
face, providing a signal that may be utilized as a cursor 
on a display unit of a computer. The code reader mode 
allows the mouse 100D to image and capture any print- 
ed symbol, for example, machine decodable symbols 
such as one-dimensional and two-dimensional codes. 
The scanner mode allows the mouse 100D to be uti- 
lized, for example, as a graphic or text scanner. 
[0113] In "video camera mode, "the mouse 1 0DD may 
be utilized simi lar to a video camera for videoconferenc- 
ing, while utilizing a computer device, not shown. The 
"video camera mode" utilizes the mouse essentially as 
a video camera, where the user may utilize it either for 
videoconferencing or for capturing images, other than 
that of the surface 130. For the video camera mode, al- 
though not shown in Figure 3D, a separate stand or tri- 
pod may be provided to support the mouse. For exam- 
ple, a tripod may be utilized on a computer display de- 
vice, or anywhere the user desires, so that the pointing 
of the sensor array may be physically stabilized for a 
videoconference. 

[0114] Alternately, another arrangement of the mouse 
100D, although not shown in Figure 3D, comprises two 
legs that maybe either folded up into the mouse housing 
106D or extended away from mouse housing 106D, 
wherein a first leg may be adjacent the left front arm 
308 L and a second leg may be adjacent the right front 



arm 30SR. The extendable legs are preferably mechan- 
ically adjustable so that the two extended legs, com- 
bined with the mouse housing 1 06D, would comprise a 
tripod-type device that may be suitably positioned by the 
5 user for use as either a stationery video camera or vid- 
eoconference device. 

[0115] In another arrangement of mouse 100D, the 
mouse housing 106D comprises a housing that is 
hinged so that it may be folded or unfolded. Although 
not shown, the hinged mouse may be suitably adapted 
to setting upon a monitor or laptop computer to provide 
stability, without the need, for example, to require a tri- 
pod-type extendable legs or an independent tripod-like 
stand. 

[011 6] Fu rther, although not shown, audio input to the 
mouse, for example, a microphone, may be input to the 
mouse housing 106D to provide an entire audio-video 
conference system, in cooperation with a computer con- 
nected to a suitable visual monitor and speaker system , 
Of course, an "audio mode" may be added as another 
setting to be triggered by the designated trigger mouse 
input device of Figure 3D. 

[0117] While all four mouse input devices 242, 244, 
246, or 248 are shown on the top surface of housing 
106D, any one or all of the mouse input devices 242, 
244, 246, or 248 may be located on the sides, or any 
oth er portion , of the mouse housing 1 06 that is p referred 
by the user. 

[0118] Another alternate use of the additional mouse 
input device is to enable the user to trigger the mouse 
device 106D to "snap," i.e., to trigger or initiate the cap- 
ture of an image. Yet another use of the additional 
mouse input device is to adjust a "zoom mode" of the 
field of view of the imager mouse 1 00D, preferably by 
utilizing the variable focus lens assembly 180 described 
herein. 

[0119] Further, in the scanner mode, the sensor unit 
120 is capable of incrementally reading printed symbols 
extending beyond the immediate field of view of the im- 
ager mouse view port area 352. For example, a lengthy 
bar code symbol, portions of a two-dimensional symbol, 
or a sequence of human readable text characters, may 
be scanned using this mode. As the imaging mouse is 
moved across the surface, images may be captured in 
real time, then passed through the sensor unit 120, and 
made capable of fitting into an electronic representation 
of the scanned image. For example, the electronic rep- 
resentation of the scanned image may be accomplished 
utilizing directional and displacement cues from the 
mouse, i.e., mouse mode function X-Y relative move- 
ment data, or form embedded codes in small sub-por- 
tions of a symbol. Either the relative position data of 
each portion, or the embedded codes, or both, may be 
utilized to scan in and assemble the entire image for fur- 
ther processing. 

[0120] Alternately, the imager mouse 100D of Figure 
3D may be implemented so that it is self -triggering." For 
example, the mouse sensor unit 120 may continue to 
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monitor the image data provided by the sensor array 122 
as the mouse 100D is moved about. When the sensor 
unit 1 20 realizes that at least a portion of the sense data 
comprises, lor example, a machine-decodable symbol, 
then the sensor unit sell-triggers to go into the code 
reader mode. Also, the imaging mouse 1 0OD may con- 
tinue to sense the area to be imaged and continue to 
provide, lor example, data resulting Irom operation in 
mouse mode simultaneously with data resulting from 
the code reader mode. In another arrangement, a time- 
sharing technique is practiced, wherein the sensor unit 
120 alternately performs, e.g., the code reading mode 
and the mouse mode, and alternates between these 
modes at a rate sufficient to be imperceptible to the user, 
such that the user believes that both modes are operat- 
ing simultaneously. Of course, the same digital image 
data may be stored, and then re-processed as many 
time as desired, to obtain results for any and all modes 
desired by the user, e.g., for approximately real time re- 
sults. 

[0121] In yet another arrangement, the self-triggering 
capability may be utilized to provide an automatic focus, 
or field of view, adjustment by the mouse 100D, rather 
than requiring an input from the user. 
[0122] In video camera mode, the mouse 1 00D is uti- 
lized with a variable focus lens assembly 1 30 that is also 
depicted in Figures 5A-B. 

[0123] The variable focus lens assembly 180 is pref- 
erably implemented with mouse 100D, as shown in Fig- 
ure 3D. A lens adjustment wheel 381 is provided for 
manual adjustment by the user on mouse 100D. The 
lens adjustment wheel 381 comprises a portion of the 
variable focus lens assembly 180. The mouse housing 
10GD is adapted to allow a portion of lens adjustment 
wheel 381 to extend outside the housing 106D. Al- 
though not shown in Figure 3D, the variable focus lens 
assembly 180 is situated between the sensor unit 120 
and the area to be imaged. The variable focus lens as- 
sembly 180 preferably also includes the sensor array 
122. However, if desired, the lens assembly 180 and the 
sensor array 122 may be physically separate. 
[0124] This preferred arrangement of the variable fo- 
cus lens assembly 180 is described in detail with Figures 
5A-B. 

[0125] Figures 5A-B illustrate a preferred variable fo- 
cused lens assembly in the pmcticeoi the instant inven- 
tion. Figure 5A illustrates a cross-sectional side view 
taken along line 5A-5A of Figure 5B. Figure 5B illus- 
trates a front view of the variable focused lens assembly. 
[0126] In Figure 5A, the variable focused lens assem- 
bly device 180 is comprised of a variable focus adjust- 
ment wheel 381, a lens housing 188, and a printed cir- 
cuit board 189. Preferably, the variable focused lens as- 
sembly 1 80 houses the sensor array 1 22, where the light 
sensing side 222 of the sensor array 122 is positioned 
nearest the focus adjustment wheel 331 . 
[0127] The lens housing 188 further comprises a first 
housing portion 188a, a second housing portion 188b, 



and a third housing portion 188c. These portions allow 
the lens housing 180 to be collapsed or extended along 
the variable focus path 1 29. Th e focus adj ustment wheel 
381 comprises a plurality of raised portions 481 and de- 
5 pressed portions 581 , so as to allow the user to friction- 
ally rotate the focus adjustment wheel 381, thereby 
moving a lens 187 either closer to or further away from 
the sensor array 1 22 along the variable focus path 1 29. 
The variable focus lens assembly 180 may be utilized 
when integrated with, for example, a mouse 1 00D Fig- 
ure 3D, to provide the user with an adjustable field of 
view or focus range. This allows the user to alternate 
between, for example, the imaging of material fibers, to 
the imaging of larger two-dimensional machine decoda- 
ble symbols, to imaging as a video camera. 
[0128] As shown in Figure 3D, by utilizing the variable 
focus lens assembly 1 80, the user may be provided pref- 
erably with a first-range line 382 and a second range 
line 384. These range lines may indicate to the user pref- 
erable focus adjustments based upon pre-set markings 
on the focus adjustment wheel 381 , respectively. Thus, 
the user may dial a focus adjustment and have a visual 
representation of the area that the sensor array 122 is 
able to image. For example, if set at the first range set- 
ting 382, the user may image anything within the bound- 
aries between the range line 382 and the sensor unit 
120. If a longer focal length is desired, the user will set 
the focus adjustment wheel 381 to a setting that corre- 
sponds to the second range line 384. Obviously, other 
ranges may be practiced with this invention. 
[0129] Flange indications may also be displayed on 
an LCD display on the mouse, or on a computer display 
device, where a graphical representation of the approx- 
imate range may be represented to the user. This dis- 
play may include, for example, the above described ref- 
erence line boundaries together with the image output,. 
The display may also include an image of a conference 
room with a maximum allowable distance away from the 
mouse in order to be included in the videoconference. 
The display may also indicate when the lens is at an 
infinity setting. 

[0130] Another alternative arrangement of the varia- 
ble focus lens assembly 180, that is not shown, is to pro- 
vide a plurality of lenses on a modified focus lens as- 
sembly. This alternate assembly would comprise a plu- 
rality of lenses that may be rotated into the variable fo- 
cus path 129, so that the user may select a variety of 
fixed focal length lenses, depending upon the mode the 
user desires. This alternate arrangement may be re- 
ferred to as a "dial-a-lens" assembly. 
[0131 ] Both the d ial-a-lens and the variable focus lens 
assembly 1 80 may alternatively be automatically adjust- 
ed and positioned, e.g., by an electric motor device, that 
is preferably miniaturized. The lens adjustment and po- 
sitioning may be triggered by a mode selection by either 
the user or the CPU. 

[0132] The variable focus lens assembly 180 and the 
lens 1 87 may be used with, for example, a CMOS array 
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that has a lens already formed as a portion of the array 
itself, thereby utilizing lens 187 as a secondary lens for 
adjusting the focal length. Also, the CMOS array may 
be provided without a lens, such that the lens 187 is the 
sole optical lens of the mouse. Alternatively, a plurality 
of lenses, not shown, may be provided in place of the 
single lens 187 illustrated in Figures 5A-B. 
[0133] A preferred implementation of the variable fo- 
cus lens assembly 1 80 is to have the lens 1 87 and sen- 
sor array 1 22 arranged such that approximately a 1/3 to 
1/2 rotation of the focus adjustment wheel 381 is ade- 
quate to change the focal length. A focal length adjust- 
ment example is from approximately 0.2 1 to 2.0, 1 and 
with further rotation may be set to infinity. 
[0134] As will be seen from the specifications of the 
examples of sensor arrays described below in Figure 6, 
a 640x480 VGA may be utilized in the practice of the 
instant invention. However other arrays may be appro- 
priate as the preferences of the user are considered. 
[01 35] A variety of lenses may be selected depending 
upon the primary field of view desired by the user. Both 
the focal length and the F-stop of the lens, or lens as- 
sembly, that is selected may allow for variations in per- 
formance for the user. 

[0136] For example, a 'short' lens may be utilized for 
imaging in the range of approximately 0.05 inches to 
0.20 inches, where the imager mouse is primarily look- 
ing down at a surface, for example, to image material 
fibers. Alternatively, a "medium" lens with a field of view 
of approximately 0.20 inches to 4 inches maybe utilized 
when the imager is primarily sensing bar codes and the 
like. Also, a "long" lens with field of view of approximate- 
ly 6 inches to 24 inches may be preferred when the im- 
ager mouse is primarily used as a video conferencing 
camera. 

[0137] Of course, these various fields of view may be 
implemented in one mouse system by utilizing, for ex- 
ample, the variable focus lens assembly illustrated in 
Figures 5A-B. This variable focus lens assembly com- 
prises an equivalent of the short, medium and long lens- 
es. Also, other alternatives, including the dial-a-lens as- 
sembly previously described, may be utilized to provide 
an equivalent of all, or portions, of the short, medium 
and long lenses, as desired. 

[0138] In one arrangement in the practice of the in- 
stant invention, a typical field of view of around 0.7 inch- 
es square may be preferred. Alternatively, another ar- 
rangement in the practice of the instant invention utilizes 
a p referred fie Id of view of approximately 1 to 1 .5 inches. 
In yet another arrangement, the area of interest is kept 
further away from the imager, but the invention utilizes 
the same optics as in the approximately 0.7 inch square 
field of view arrangement. Each of these arrangements 
may be provided with an adjustable or fixed lens, that 
may result in a system that is able to resolve approxi- 
mately a minimum often mil features, depending upon 
the imager chosen. 

[0139] The preceding paragraph describes an ar- 



rangement of a useful field of view when reading optical 
codes with 10 mil features. However, one "ru le of thumb" 
for determining the size of the field of view is to have at 
least 1 .5 pixels per minimum element in the optical code. 

5 So, for example, the 640 by 480 pixel imager described 
herein may have a relatively large field of view, for code 
reading, of up to, e.g., about 4.3 by 3.2 inches, when 
reading 1 0 mil codes. However, this field of view may 
be diminished if the imager is not operating under ap- 

10 proximately ideal circumstances, such as low noise, 
minimal motion blur, and the like. Because such a rela- 
tively large field of view is not always desired or practi- 
cal, particularly when in contact with the surface, a 
smaller imager may be used, and/or the optical magni- 
fy fication may be changed so that finer codes may be 
more easily read. For example, the user may wish to 
resolve a minimum of approximately 7 mil features in- 
stead of about 10 mil features. 
[0140] As to motion blur, the approximate maximum 

20 allowable exposure time setting, or alternately, the ap- 
proximate maximum illumination strobe duration set- 
ting, may be based upon, or relation ally calibrated, to 
the expected, e.g., approximate maximum velocity of 
the mouse. For example, if the expected approximate 

25 maximum velocity of the mouse is, e.g., 50 inches per 
second, then an approximate maximum exposure time 
may be set at, e.g., approximately 100 microseconds. 
These maximum settings may be desired to limit the rel- 
ative motion to an approximate maximum motion, e.g., 

30 of approximately 5 mils during the image integration 
time, based upon an example similar to the feature res- 
olution examples described above. Of course, other al- 
ternate limits may be set, depending upon, e.g., the fea- 
ture resolution desired, and/or the sensor array utilized. 

3S [0141] Figures 4A-D illustrates both some examples 
of machine decodable symbols and of some alternative 
methods of capture of a machine decodable symbol. 
Figure 4A illustrates an example of a conventional one- 
dimensional bar code. Figures 4B-D illustrate some al- 

40 ternate techniques for capturing a two-dimensional ma- 
chine decodable symbol. 

[0142] A more detailed description of one-dimension- 
al bar codes, e.g., as illustrated in Figure 4A, and some 
alternate signal processing techniques for recognition 
45 of one-dimensional barcodes, are more filly described 
in U.S. Patent No. 5,446,271, to Cherry, et. al., and is 
incorporated by reference as though fully set forth here- 
in. 

[0143] Some other techniques for signal possessing 
so of machine decodable symbols are more fully described 
in U.S. Patent Nos. 5,825,006 to Longacre, Jr., et. al., 
5,513,264 to Wang, et al., and 5,296,690 to Chandler, 
et. al., and all three of these patents are incorporated 
by reference as though fully set forth herein. 
55 [0144] As shown in Figure 2B, the two-dimensional 
symbol 161 may be imaged or scanned in its entirety if 
its size falls within the focal length of the mouse 100. 
However, if the machine decodable symbol is either 
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larger than the local length, or difficult to capture in its 
entirety, or alternatively, if only a portion of the symbol 
is initially encountered by the imaging mouse, then a 
technique such as is illustrated in Figure 4B may be uti- 
lized. 

[0145] For example, the user begins imaging the sym- 
bol at the start position 1 34, and then the user moves 
the imaging mouse along the approximate imaging path 
1 35. In a preferred operation, the user moves the imag- 
ing mouse across the symbol, first traveling on a first 
imaging path 135a1, then reversing direction at a first 
imaging path directional change 135c1, and continuing 
therefrom on a second portion of the imaging path 
135b1. The user again reverses direction at a second 
imaging path directional change 135c2, so as to travel 
along a third portion of the imaging path 1 35a2. The user 
again changes direction at a third imaging path direc- 
tional change point 135c3, so as to travel along a fourth 
portion of the imaging path 135b2, where again the user 
changes direction at a fourth imaging path directional 
change 135c4, so as to continue along a fifth portion of 
the imaging path 135a3. The user may continue in this 
manner for as many times as required so as to image 
the entire machine decodable symbol. Upon completion 
of the plurality of portions of imaging paths, the user 
would arrive at an end position 1 36. 
[01 46] Figu re 4C illustrates a display device 1 90, that 
may be located, for example, on the imaging mouse it- 
self, or as a part of a computer display device that may 
be utilized with the imaging mouse. A dashed line illus- 
trates the real-time location of the mouse along a portion 
of the imaging path, in this case the third portion of the 
imaging path 135a2. As the user moves the mouse 
along path 135a2, preferably the user is able to view on 
the display device 190, and preferably in real time, the 
machine-decodable symbol 161 as it is constructed, i. 
e., as it is sensed and imaged by the imaging mouse. 
[0147] Alternatively, the user may choose not to follow 
an "orderly" path such as the imaging path 1 35 illustrat- 
ed in Figure 4B. Instead, the user may choose to follow 
a less orderly path, for example, similar to the path 1 38 
illustrated in Figure 4D. 

[0148] There are at least two reasons why the user 
may wish to follow a less orderly path. First, the user 
may find it more convenient not to follow quite such an 
orderly path for ergonomic reasons. 
[0149] Second, and as illustrated in Figure 4D, the us- 
er may or may not have attempted to follow an orderly 
path, but was not able to capture the entire machine de- 
codable symbol 161. For example, the user may have 
found that certain Void' areas 261 resulted. But the user 
preferably desires to complete the two-dimensional 
symbol 161 so that the voids 261 are substantially elim- 
inated. 

[0150] One technique to reduce or eliminate the voids 
261 is to utilize a "paintbrush" or "reverse erasing" tech- 
nique. A paint function, or 'paint mode" is enabled or 
triggered. The paint mode may be triggered either au- 



tomatically or through a user input, for example, utilizing 
one of the mouse input devices of Figure 3D to trigger 
the reverse erasing function 169. 
[0151] The user positions the mouse appropriately on 

5 the work surface either near, or upon, the two-dimen- 
sional symbol 161. The user may, e.g., manually trigger 
that the user is at a preset start position 234. The start 
position 234 is preferably indicated on a display, to help 
guide the user. Alternatively, the processor, e.g., utilizing 

10 software, may determine that the symbol is within imag- 
ing range again, and update the cursor based upon, e. 
g., a partially recognized portion of the symbol with the 
voids. 

[0152] Then, e.g., by either a user trigger input, or by 
a self -triggering operation, the cursor is placed into a 
reverse erasing symbol 169 configuration. The relative 
representational oscillating, or back and forth "erasing 
movement" is represented by the motion arrows 269. 
The cursor movement represented on the display de- 

20 vice 1 90 is relative to the movement by the user of the 
imaging mouse on the surface area to be imaged, where 
the actual two-dimensional symbol 1 61 is located. Then, 
from near the start position 234, the user essentially uti- 
lizes the imager mouse similar to an eraser on a chalk- 

25 board, where the user attempts to move the cursor rep- 
resented by the reverse eraser 169, and the motion ar- 
rows 269, over the void areas 261 so as to complete the 
two-dimensional symbol image 161 . At some point, the 
user is able to complete the reverse erasing procedure 

30 and arrives at the end position 236 as shown in Figure 
40. 

[01 53] In another arrangement the void areas 26 1 are 
preferably represented by one or more contrasting 
colors, with respect to the color or colors representing 

3S the portion of the 2D symbol 161 that previously has 
been captured properly. In another alternative, e.g., a 
gray scale contrast, or differing patterns, may be utilized 
to provide the visual differentiation between the void ar- 
eas 261 and the properly scanned areas of the 2D sym- 

40 bol 161. 

[0154] The display 190 may comprise one of a 
number of alternatives. For example, the display may 
be a type of 'thumbnail' sketch. The thumbnail sketch 
may provide adequate information for the user to dis- 

45 cern that the symbol has been recognized and captured 
by the system, without unduly taxing either the sensor 
unit, or the processing unit that will be described later. 
Alternatively, either a graphics display window, or the 
entire display, may be utilized with a high resolution, par- 

50 ticularly when the reverse erasing function is in 
progress. Another alternative is to provide an audio sig- 
nal of either voice synthesis or of a recognizable tone 
or sound that indicates to the user whether a symbol has 
been captured properly or needs further work, for exam- 

55 pie, a symbol needing the reverse erasing function. 
[0155] Another alternative is to provide a library of 
symbols, as previously described for the sell-learning 
technique. The sensor unit 120 may utilize the library of 
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symbols to interpolate, suggest, or even "guess" at sym- 
bols, once at least a portion of the symbol has been im- 
aged. In this case, the reverse erasing or painting func- 
tion may only need to be partially used, until the sensor 
unit determines that it has enough data to recognize the 
symbol accurately. Alternatively the display device may 
display a list for the user to choose from. 
[0156] Also, as previously described for the direct 
symbol recognition, a self -learning type of implementa- 
tion is possible in recognizing machine-decodable sym- 
bols in conjunction with the reverse erasing or painting 
function. The sensor unit 120 may also utilize a self- 
learning function together with the painting function for 
text recognition. 

[0157] Also, a linear CCD may be utilized in an alter- 
nate assembly technique. For example, a line by line 
assembly may be performed with this linear sensor. 
[0158] A lookup table, as previously described, may 
also be utilized to provide an estimate of the scale of the 
bar code to be scanned, and may be combined with a 
partial scan of the symbol. 

[0159] Although one-dimensional and two-dimen- 
sional code symbols have been used in the illustrations 
in Figures 4A-D, any code symbols or other graphical 
images may be imaged with the above-described cap- 
ture and reverse erasing functions. For example, a 
maxi-code that has a 'bull's eye" in the middle of the 
code may be utilized. 

[0160] Of course, either the thumbnail sketch or full 
resolution display of the two-dimensional symbol, for ex- 
ample, may be activated automatically by the sensor 
unit, as opposed to requiring a user input from a mouse 
input device. 

[0161] And, for example, a code symbol may be uti- 
lized for data entry or as part of a reference lookup table 
to provide significant data availability to the user in con- 
junction with utilizing the imaging mouse. 
[0162] An example of a device utilized in the applica- 
tion of a library or look-up tables, for easier recognition 
or assembly of an image, or selection or "guessing" by 
the processor, or the user via a display, includes the Log- 
iCORE One Dimensional RAM-Based Correlator by 
XILINX, Inc., San Jose, CA, for one-dimensional sym- 
bols. A Two Dimensional RAM-Based Correlator is also 
available from XILINX, Inc., for two-dimensional sym- 
bols. 

[0163] Further, e.g., a two-dimensional symbol may 
be previously encoded with re-assembly instructions or 
embedded codes located throughout the symbol. For 
example, the symbol may be divided up into sectors, e. 
g. , 1 6. Each sector may comprise a predefined area, or 
portion, including a predefined boundary of the sector 
portion of the symbol , and the embedded code data may 
be utilized for pattern matching or correlation. The 
unique assembly instruction data embedded in the sym- 
bol would allow a processor to determine which portion 
of the symbol that the sensed portion belonged to. This 
would also allow for a more accurate re-assembly, and 



a check for correctness. Also, the symbol may be pre- 
divided, e.g., corresponding to the embedded assembly 
reference codes. If only a portion were sensed during 
any one scan, then only the portion that included the 

s symbol's predetermined sector boundaries need be re- 
tained for assembly with the other portions to be 
scanned in the next scanning pass. This may also re- 
duce the processor overhead required in an assembly 
process of a partially scanned symbol, particularly when 

10 combined with a look-up table technique. 

[0164] Re-assembly instructions or embedded codes 
located in the symbol may also be understood from, for 
example, the row numbers and codeword clusters uti- 
lized in PDF417 symbols, or the internal data regions 
and finder patterns utilized in DataMatrix ECC200 sym- 
bols. 

[0165] The auto-matching described above may be 
performed with the image processor FPGA and Corre- 
lator, described above, and an increased memory. How- 

£0 ever, for a complete self -learning technique, such as 
storing and utilizing the features of a surface repetitively 
encountered, and for improved recognition of a surface, 
e.g., when the invention is utilized in a self-learning 
mouse mode, then a f ul I microprocessor may be util ized. 
[0166] It is understood by one skilled in the art that 
the imager mouse, by having an X-Y relative movement 
function, i.e. the mouse mode, may construct, e.g., a 
two-dimensional symbol 161 as illustrated in Figures 
4B-D in an accurate manner. This accurate construction 

30 is facilitated because the X-Y coordinates of each por- 
tion of the symbol may also be gathered as the imager 
is passed over the symbol in the code reading mode or 
in the paint mode. 

[0167] In an alternate arrangement, utilizing a CMOS 

3S array, the mouse and code reader, or mouse and paint 
modes, for example, appear to be active essentially at 
the same time. The invention may gather a data set at 
a rate of approximately between one and 30 times per 
second. The digital data for this CMOS array arrange- 

40 ment, or for the CCD arrangement described herein, 
may be stored and reprocessed as may times as de- 
sired, so that the same data may provide results for an- 
ything between one and all of the modes available. 
Thus, the various modes may be made to appear to be 

45 running simultaneously to the user. 

[0168] In another arrangement, two linear high speed 
CCD imagers and a relatively fast processor are utilized, 
so that about 1 000 to 2000 scan lines per second may 
be realized. These scan lines may be analyzed, e.g., in 

so a mouse mode, or assembled into an image, e.g., to rec- 
ognize a two-dimensional symbol. Also, this CCD ar- 
rangement may be provided in a "crosshair" type con- 
figuration. 

[0169] For example, an alternate cross-hair design of 
55 a CMOS sensor array may actually improve imaging for 
relative motion yet require less actual processing 
throughput. Also, the image data of the area to be im- 
aged may be utilized similar to a fax machine output 
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where the representative image is assembled, tor ex- 
ample, line by line and only alter it is assembled does 
the processor act to recognize a machine decodable 
symbol. 

[0170] Figure 6 illustrates a system block diagram ol 
an imager mouse that utilizes a USB type controller. The 
elements comprising Figure 6 illustrate an arrangement 
of the imager mouse 1 00 as il lustrated in Figures 2A-3C. 
[0171] Generally, Figure 6 illustrates a CMOS sensor 
array 122 that provides image data to an image proces- 
sor 1 91 . The CMOS sensor array 122 may also provide 
data to a USB controller 1 94. The image processor 1 91 
utilizes a memory device 1 92. The image processor 1 91 
may also provide data to the USB controller 1 94. USB 
controller 1 94 controls the data flow between either the 
array 122 and processor 191 , and at least one of a CPU 
1 98, an I RDA type connection 196, a display device, and 
any mouse input devices. 

[0172] In another arrangement, the mouse electron- 
ics may comprise a controller of some sort, e.g., a USB 
controller, or some other controller, or an additional con- 
troller for image processing power. The electronics may 
also comprise a memory device, e.g., a FLASH, RAM, 
or ROM type memory device, that may also, e.g., run 
the controller code. Further, the electronics may also 
comprise a logic device, e.g. , a microprocessor, or some 
other related or similar logic technology, an array sen- 
sor, and an output device . Alternately, one or more of 
these electronics may be "inside" or integrated with the 
controller. 

[0173] Figure 6 illustrates an image processor 191 
performing either a partial processing, or a complete 
processing by including, e.g., a video driver, of the data 
signal provided from the CMOS sensor array 1 22. The 
user choice of providing either partial processing or 
complete processing by the image processor 191 de- 
pends at least in part upon both the type of processor 
implemented and how the user desires to utilize the 
processor 191. Then, either the partially processed, or 
completely processed, data signal is transmitted to the 
CPU 198 for either final processing, or for utilization, re- 
spectively, via the USB controller 194. 
[0174] Thus, the complete interpretation of data, for 
example, in the form of decoding a one or two dimen- 
sional barcode, or the interpretation of text such as per- 
forming OCR on the scanned text, and any other various 
data processing related to utilizing the instant invention, 
may be completely performed onboard, i.e., inside, the 
optical code reader mouse device, if desired. 
[0175] However, a number of alternate arrangements 
are possible, as are described herein. 
[0176] As illustrated in Figure 6, reflected light 204cc 
is received by the CMOS sensor array 122. Upon re- 
ceiving the reflected light 204cc, the sensor array 122 
generates an electrical data signal along line 204j that 
is representative of the reflected light 204cc. The data 
signal maybe provided to an image processor 1 91 along 
line 204j, or directly to a controller 1 94 along line 204e. 



The sensor array 122 data signal is transmitted in timing 
coordination with the system clock provided on line 204b 
and the pixel clock provided on line 204f. 
[0177] Upon receiving the timed data signal along line 
s 204j, the image processor 1 91 may, for example, store 
the data signal in memory device 1 92, or process the 
data signal and then store the results in memory device 
192. 

[0178] Alternately, the image processor 191 may re- 
10 ceive the data signal along line 204j and transmit it to 
the controller 194 along the signal line 204s, without first 
processing the data signal. 

[0179] The image processor 1 91 may process the da- 
ta signal in accordance with one or more previously de- 

is scribed modes selected by the user. The user, e.g., may 
define the active mode or modes, as wel I as provide oth- 
er user input, via at least one of a trigger signal line 343, 
a right mouse button signal line 342, a center mouse 
button signal line 344, or a left mouse button signal line 

« 346. 

[0160] Alternately, user inputs may be provided via 
CPU 198. CPU 198 communicates with the USB con- 
troller 194 utilizing the USBD+data signal line 204x and 
the USBD- data signal line 204w, or alternately by a 

25 wireless technique previously described. 

[0181] These user inputs may be communicated by 
the controller 1 94, e.g., along the fast read data signal 
line 204t to the image processor 1 91 . The image proc- 
essor 1 91 then initiates the mode or plurality of modes 

30 based upon these user inputs. 

[0162] Upon initiating the operative mode, the image 
processor 1 91 processes the data signal supplied along 
line 204j by utilizing computer software and/or firmware. 
Known processing techniques may then be utilized to 

3S provide the previously described display outputs. 

[01 83] As shown in Figu re 6, the image processor 191 
may comprise at least one of a low power field program- 
mable gate array ("F PGA"), a cool complex programma- 
ble logic device ("cool CPLD') or an electronically eras- 

40 able programmable logic device ("EPLD"). An example 
of an EPLD is the XI LI NX XC951 44XL device. Alternate- 
ly, the image processor may comprise at least one of 
the EPLD, the FPGA, the cool CPLD, a microprocessor, 
or some other related or similar logic technology. 

45 [0164] The mouse may comprise an enhanced capa- 
bilities port ("ECP B ) (Microsoft) driver for the transfer of 
data, e.g., for sending video data to the host. However, 
because of logistical concerns with the ECP, e.g. the 
number of conductors and lack of a power supply, the 

so ECP is not currently utilized in a preferred embodiment. 
[0185] The processed data, and/or video output, is 
then provided to the USB controller on the data signal 
line 204s. The USB controller 194 then provides this 
processed data or video signal as requested, for exam- 

55 pie, to the CPU 198 by any of the transmission tech- 
niques previously described. The USB controller may 
alternately obtain serial data signal 204e, i.e. "raw 1 da- 
ta, from the CMOS sensor array 122, or raw or partially 
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processed data from the processor 1 91 , and provide it 
as requested, preferably to CPU 198 for processing. 
[0186] The image processor 1 91 may also provide an 
LED control signal 272 for controlling an LED illumina- 
tion source and pointing device 372. The illumination 
and pointing device 372 is optional in the practice of the 
invention. The illumination and pointing device 372, in 
an illumination mode, may comprise, for example, the 
strobed LED light source previously described, or may 
control an array of LEDs that are able to illuminate at 
varying intensities, depending upon the state of the LED 
control signal 272. 

[0187] Alternatively, the illumination and pointing de- 
vice 372 may be utilized to focus light in order to provide 
a type of pointing indicator in a "pointing mode. 1 In the 
pointing mode, the approximate area to be imaged 1 32F 
is distinctly lighted by the imager mouse 100. The point- 
ing mode is utilized by the user in aiming and positioning 
the imager mouse 100, where the pointing mode ena- 
bles the user to more easily identify the approximate ar- 
ea that the imager mouse 100 may image or capture. 
[01 88] Also, an LCD viewf inder 290, as previously de- 
scribed for mouse 100, may be controlled by the proc- 
essor along the LCD Driver Signal Line 204ee. 
[0189] The sensor array 1 22 is a CMOS sensor in the 
arrangement illustrated in Figure 6. A number of sensor 
arrays may be utilized. For example, an OmniVison 
OV71 10 imager, a Motorola MCM20014/MCM20008 im- 
ager, a Photobit P300 imager, a TOSHIBA TCM5003D 
imager, or an HP 2001 imager, are some of the imagers 
that may be utilized. 

[0190] Low resolution and low cost imagers that may 
be utilized in the practice of the invention, for a mouse 
mode arrangement with some limited bar code detec- 
tion, includes the BVL and Rockwell Rp0352 imagers. 
Another alternative comprises a Rockwell R11024AB 
CMOS imager with a USB companion chip. 
[0191] Of course, these imagers may not be either 
pin-for-pin compatible, or function for function compati- 
ble, with the sensor 122 that is illustrated in Figure 6, 
but they may be adapted for utilization in the instant in- 
vention through know techniques. Thus, Figure 6 illus- 
trates but merely one example of how to connect an im- 
ager to a USE controller to practice the instant invention 
of an imager mouse, when at least some image process- 
ing is performed internal to the imager mouse device 
100. In other alternate arrangements, the processing 
may be performed outside the imager mouse device 
100. 

[0192] As illustrated by the examples herein, the sen- 
sor array 1 22 may be a monochrome type sensor, or the 
sensor array 122 may be implemented as a color sen- 
sor. 

[0193] The sensor array 122 may be implemented uti- 
lizing a commercially available sensor array. A currently 
available sensor utilizes a rectangular pattern type of 
sensor array. However, in some cases, a cross-hair pat- 
tern may be preferable for detecting motion, especially 



when the imager mouse is primarily used for detecting 
relative motion between the mouse housing 1 06 and the 
surface 130. 

[0194] Another advantage of the sensor array 122 is 
5 that one dimensional bar codes do not need to be 
aligned, as is required when utilizing a laser type scan- 
ner. Thus, the imager mouse is capable of capturing ma- 
chine decodable symbols "on the fly, " unl ike convention- 
al optical mice or laser bar code readers. 
[0195] An electronic shutter is preferably provided. 
However, a shutter device may be implemented exter- 
nally, for example, in the variable focus lens assembly 
180. The shutter may be either a mechanical or an elec- 
tronic device. 

[0196] Also, a "zoom mode" may be provided for uti- 
lization with the imager mouse, where a zoom view may 
be displayed, either on the mouse LCD display or on the 
computer display device. This would provide a zoom 
type function for utilizing the mouse as a zoom in and 
out imager. 

[0197] To activate the zoom mode, any of the input 
devices described herein may be utilized. Thus, e.g., the 
zoom in and zoom out type functions may be controlled 
by the user utilizing one or more of the mouse buttons. 
[0198] For example, the user may utilize the four but- 
tons of the mouse 100D of Figure 3D, namely, the four 
mouse input devices 242, 244, 246, and 248, respec- 
tively. The previously designated trigger button for alter- 
nating modes may be utilized by the user to trigger the 
zoom mode. Then, a left and right button maybe utilized 
to zoom in and out, respectively. 
[0199] The CMOS sensor array 122 receives a sys- 
tem clock signal along line 204b and a 3.3 volt source 
along line 204a. The CMOS sensor array 122 also re- 
ceives light 204cc. The CMOS sensor array 122 may 
receive, as an input, a scan-enable signal along line 
204c, a serial-clock signal along line 204d, and may 
transmit or receive parameter setting signals along line 
204e. In one arrangement, there may be, e.g., about 50 
parameter settings available. For example, the dataS 
signal along line 204 e may be utilized to send these pa- 
rameters to the imager, and for reading back the settings 
that are in place. The data signal along line 204e, when 
provided by the CMOS sensor array 1 22, is similar to 
the data signal along line 204j that may be provided by 
the CMOS sensor array 1 22 to the image processor 1 91 . 
[0200] The image processor 191 also receives a line 
valid signal along line 204i, a frame valid signal along 
line 204h, a shutter open signal along line 204g, and a 
pixel clock signal along line 204f, all from the CMOS 
sensor array 122. Additionally, the image processor 191 
may directly receive, e.g., a 5 or a 3.3 volt source along 
line 204k, and the system clock signal along line 204b. 
[0201] The line valid signal along line 204i is for hor- 
izontal synchronization, while the frame valid signal 
along line 204h is for vertical synchronization. 
[0202] Also, the data signal along line 204] is illustrat- 
ed as an 8-bit signal line. However, this, and any of the 
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illustrated S-bit signal lines o1 Figure 6 may be expand- 
ed, e.g., to a 10 or 12 bit signal line with the appropriate 
sensor, processor, and bus line, if desired by the user. 
[0203] The voltages specified in this system block di- 
agram, as illustrated in Figure 6, are representational in 
nature and other voltages may be utilized in the practice 
of the instant invention. For example, a CMOS sensor 
array 1 22 utilizing a voltage lower than 3.3 vofts may be 
preferable, depending upon either the user's preferenc- 
es, or the imager selected. 

[0204] The image processor 191 preferably has ac- 
cess to a memory device 192, for example, an SRAM 
(512Kx8). A minus-chip select signal along line 204m, 
an address signal along line 204n, a minus-read signal 
along line 204q, and a minus-write signal along line 
204r, are provided between the image processor 1 91 
and the memory device 1 92. The image processor 1 91 
may also bi-directionally communicate an Rdata signal 
along line 204p. 

[0205] Memory device 1 92 may be utilized to hold at 
least one entire video image as each video image is 
processed. The memory device 1 92 may be enlarged, 
e.g., if desired to be utilized to hold two or more entire 
video images. 

[0206] The data signal along line 204s may be, e.g., 
data of an equivalent type and may be of a different time 
period, or data of a completely different type, to the data 
of the data signal along line 204p,. 
[0207] Further, another arrangement may provide for 
an expanded memory device 192 capacity so as to pro- 
vide a mass storage capability, for example, video im- 
ages that may be later downloaded from the imaging 
mouse to, for example, a CPU. Additionally, the mass 
storage capability may also be implemented for storing 
a plurality of scanning or code symbol data for later 
processing or downloading. Expanded memory may al- 
so be provided to enhance the self -learning and look-up 
table techniques previously described. 
[0208] For example, either a digital signal processor 
("DSP") type device, or digital processing implemented 
in a software technique, may be utilized in the practice 
of the instant invention. Either may be utilized to provide 
complete on-board processing within the mouse itself. 
[0209] In one arrangement a plurality of mouse input 
devices, providing a plurality of user input signals, are 
connected to the controller 194, For example, as illus- 
trated in Figure 6, a left mouse button signal along line 
346, a center mouse button signal along line 344, a right 
mouse button signal along line 342, and a trigger signal 
along line 348 are provided as inputs to the USB con- 
troller 1 94. 

[0210] It will be understood that it is not required to 
provide tor four physical mouse input devices, e.g., four 
mouse buttons on the mouse housing. Instead the in- 
stant invention may utilize, for example, a "trigger but- 
ton" that instead comprises utilizing any two of the three 
mouse buttons of Figures 1 A-3C together, at the same 
time, to initiate a trigger signal. Another alternative is to 



provide for a trigger signal upon the "double clicking, 1 
or "multiple clicking" of any one of the buttons provided 
on a mouse housing. Also, the input device for the trig- 
ger signal line 348 may, for example, be a fourth mouse 
5 button, or a microswitch utilized to detect a mouse to 
surface non-contact condition. 

[0211] The raw data signal along line 204j to the im- 
age processor 1 91 may be provided at full resolution, or 
partial resolution, depending upon the operational mode 
10 of the mouse. For example, upon a trigger input by the 
user, the processor may receive either full, or partial, 
resolution image resolution data. As processors be- 
come ever faster and more efficient, a mouse may al- 
ternately be configured to continuously provide full res- 
olution data to the processor. 

[0212] Of course, any mouse button or other mouse 
input device may be utilized to trigger any of these 
events or modes of the mouse. Also, for example, driv- 
ers may be written and installed with the mouse in the 

so mouse processor, or alternatively on a CPU in commu- 
nication with the mouse, that use an interception tech- 
nique to alter the function of the button or input device 
from what is normally the case, to the unique use for 
that mode or modes of the imager mouse as practiced 

25 in the instant invention. These intercepts may be user 
defined or predefined and supplied by the user as is de- 
sired. 

[0213] In order to reduce the demand on processor 
1 91 , for example, the imager mouse may provide the 

30 mouse mode operation of X-Y translational movement 
detection lor a cursor signal by merely detecting a rela- 
tively few edges in images and then track the movement 
of at least one of those edges from frame to frame. Al- 
ternatively, if the mouse is in an image capture-type 

3S mode, such as a scanner mode, or code symbol mode, 
the mouse may merely compress the image data and 
transmit it to a remote CPU 1 98, for either a continuation 
of, or complete, processing by the CPU 198. 
[0214] However, the mouse image processor 191 

40 may process the data and provide a video signal along 
signal line 204ee for use on the mouse display LCD 
viewfinder290, oralternatively, or even in parallel, trans- 
mit a video signal to the CPU 1 98 for display or storage. 
In either case, the data may be compressed image data 

45 that is transmitted or provided as an output from the 
mouse, or may be video data that may also be com- 
pressed, for example, by utilizing an MPEG, MPEG-2, 
or JPEG compression technique. Compression tech- 
niques may be accomplished by a software implemen- 

50 tation or by an electrical hardware implementation. 
[0215] In transmitting any of these forms of data, a 
controller such as the USB controller illustrated in Figure 
6 may be utilized. One example of a USB controller is a 
high-speed controller model 8X931 Ax. Another ar- 

55 rangement of a controller comprises an Anchor brand 
USB controller, model number AN2135SC. 
[0216] Also, a low speed controller may be utilized, 
particularly when the invention is utilized in mouse 
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mode, or a relatively limited amount of data is provided 
as an output of the mouse device. However, for symbol 
imaging a high speed controller is preferable. 
[0217] It will be obvious to one skilled in the art, that 
utilizing a USB interface teaches bidirectional commu- 5 
nications that may allow for a number of configuration 
capabilities that may be initiated by, e.g., the host or the 
CPU. Any of these configuration capabilities may be in- 
itiated by the host in addition to, or in replacement of, 
the various physical trigger button arrangements that 10 
are described herein. 

[021 8] The imaging mouse device may be alternately 
implemented in a manner such that the device con- 
sumes approximately 100 mA or less. However, in an- 
other arrangement, the mouse device merely need con- is 
sume less than approximately 500 mA, because this is 
typically made available from a conventional USB con- 
troller. Preferably, the USB controller operates at ap- 
proximately 5 volts, and provides the same voltage to 
the image processor and other components. Although 20 
not shown, a DC-DC regulator or a step-down trans- 
former may be utilized to provide the 3.3 volts that is 
currently illustrated for the CMOS sensor array 122 and 
processor 1 91 . 

[021 9] An advantage of utilizing the Anchor USB con- 25 
troller is that at least some data may be processed on 
board the USB controller itself. Also, the Anchor USB 
controller provides for memory on board the controller. 
This helps enable the Anchor USB controller to, for ex- 
ample, provide essentially real time updates of, for ex- 30 
ample, changes in the drivers utilized for the various 
mouse buttons, the drivers utilized for various modes of 
the imager mouse, and the like. In other words, the im- 
ager mouse need not have any preset drivers installed 
but instead may obtain them in approximate real time 3S 
via the Anchor USB controller. 

[0220] In Figure 6, the USB controller 1 94 generates 
the scan enable signal along line 204c, and the serial 
clock signal along line 204d, from the CMOS sensor ar- 
ray 122, along with a bidirectional serial data signal w 
along line 204e between the sensor array 1 22 and the 
USB controller 194. Thus, the USB controller 194 may 
provide raw sensor imagery data from the mouse as an 
output. 

[0221] Also, the USB controller 194 may be bi-direc- 45 
tionally coupled with processor 1 91 utilizing a processor 
data signal along line 204s. The USB controller 194 may 
transmit, to the processor 191, a minus-fast read data 
("-FRD") signal along line 204t. Also, the USB controller 
1 94 may receive, from the image processor 1 91 , an so 
over-exposure signal along line 204u and an under-ex- 
posure signal along line 204d. 

[0222] Thus, the USB controller 194 is enabled to pro- 
vide processed image data as an output, for example, 
to an on-board LCD viewfinder 290, along a signal line 
not shown, or to an external CPU 1 98. In connecting the 
USB controller 194 with an external CPU 198, a USBD+ 
data signal along line 204x and a USBD- data signal 



along line 204 w is provided as one arrangement of a 
two-way data communication link between the USB 
controller and the external CPU 198. The USE controller 
may also receive a Vcc source along line 204aa and a 
ground reference signal along line 204bb from the CPU 
198. 

[0223] In another arrangement, a controller, prefera- 
bly a USB controller, utilizes at least one of an iso- 
chronous, a bulk mode, or an interrupt transfer tech- 
nique. 

[0224] Figure 6 illustrates an alternative communica- 
tion connection between the USB controller and any ex- 
ternal device. The USE controller 1 94 is shown connect- 
ed to the CPU 198, by utilizing an IRDA 196 infrared 
connection. For example, a small window port, that is 
not shown on mouse device 1 00 or 1 0Od, may be utilized 
for an infrared output signal along path 204z and an in- 
frared receive signal along path 204y, connecting the 
controller 194 with the CPU 1 98. 
[0225] The infrared IRDA 196 communication link 
may, for example, provide data at 4 megabits per sec- 
ond. 

[0226] Of course, any type of radio frequency, elec- 
tronic, light or acoustic communication technique may 
be utilized in the practice of this invention, For an exam- 
ple describing an acoustic technique, see U.S. Patent 
No 5,665,956 to La, et. al., that is also incorporated by 
reference as though fully set forth herein. Thus, this 
wireless alternative is not limited to an implementation 
of an IRDA communication technique, but may com- 
prise any radio, optical, or acoustic links, instead of 
wires or signal lines. 

[0227] In Figure 6, the signal lines with a 'minus," or 
■-■ sign, indicate "active low" signal lines. Also, in the 
preferred embodiment, the chip select signal 204m is 
not utilized, but may be provided for the user that wishes 
to implement power conservation techniques. 
[0228] In summary, the imager mouse, as illustrated 
in Figure 6, is a solid state sensor array 122 connected 
with an image processor 191 having a memory device 
1 92, and are connected to a controller 1 94. In another 
arrangement, the sensor, processor, memory and con- 
troller are all located on one electronic device, i.e., one 
■chip. 1 

[0229] Figures 7A-B illustrate block diagrams of alter- 
nate arrangements that may also be utilized in the prac- 
tice of the instant invention. 

[0230] Figure 7A illustrates a block diagram of a pre- 
ferred logic arrangement in the practice of the instant 
invention. Mouse housing 106 contains mouse internal 
bus 104 connecting the sensor unit 120 with the mouse 
tail bus 102. The mouse tail bus provides an output data 
port for mouse 100. 

[0231] The sensor unit 120 is further comprised of a 
logical control unit 320 and a logical imaging unit 220. 
Although not shown, the imaging unit 220 further com- 
prises a sensor array, for example, similar to the sensor 
array 122 of Figure 6. Also not shown, the control unit 
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320 of Figure 7A may comprise a controller 194 and an 
image processor 191, if desired. 
[0232] In the arrangements of Figures 7A-B, the sen- 
sor array of the imaging unit 220 is considered to be 
pointing along a horizontal axis by receiving light that 
travels along light path 125. However, a logical arrange- 
ment similar to the devices illustrated in Figures 7A-B, 
may be utilized with an alternative physical arrangement 
that is considered to be pointing along a vertical axis. 
[0233] A display signal along line 390 may be provid- 
ed to connect an LCD display 290 with the sensor unit 
120. 

[0234] A CPU 1 98 may be connected via the mouse 
tail bus extension 202 along with a CPU display 298. 
The sensor unit 1 20 provides a signal via the control unit 
320. This signal may be either in final form, or in need 
of processing, as desired, so that the CPU 1 98 may pro- 
vide the display 298 with a video signal. 
[0235] Figure 7B illustrates an alternative logical ar- 
rangement in the practice of the instant imaging mouse 
invention. In Figure 7B, the control unit 320 is logically 
removed from the mouse 1 00 and instead, in this exam- 
ple, co-located with the CPU 198. In this arrangement, 
either raw data, or partially processed imaging data, 
may be provided by the sensory unit 120 to the control 
unit 320 for use by the CPU 198. 
[0236] In this arrangement illustrated in Figure 7B, the 
sensor unit 120 may comprise, for example, the USB 
controller 1 94 as illustrated in Figure 6. The control unit 
320 may comprise, for example, the image processor 
191 as illustrated in Figure 6. 

[0237] Other alternative arrangements of the instant 
mouse invention are many of which but of a few follow. 
[0238] First, a track ball mouse similar to the mouse 
100 illustrated in Figures 1A-C. but with the imager 
pointing upward toward the user, may be realized. The 
mouse is utilized as a conventional mouse with a fric- 
tionally engaged track ball. However, the Imager is uti- 
lized in a similar manner to, e.g., a bar code scanner in 
a checkout stand of a grocery store, where the area to 
be scanned is passed over the sensor array, rather than 
the sensor array relatively moved to the area to be 
sensed. Once the imaging of, for example, the barcode 
and like has been completed, the user may continue to 
use the trackball portion of the mouse for X-Y cursor in- 
formation. 

[0239] Another embodiment of the instant imaging 
mouse invention is illustrated in Figure 8. In Figure 8, a 
CPU 398 comprises an input pad 240, and a display 290 
connected via internal mouse bus 104 to sensor unit 
120. Sensor unit 120 further comprises a sensor array 
322. The sensor unit 120 is preferably connected to an 
alternate imaging unit input port 380 via an alternate port 
bus 304. The alternate imaging unit input port 380 may 
further comprise a second sensor array to image from 
the alternate imaging unit port area 254. 
[0240] Alternately, the same imager may be utilized 
for both the alternate imaging input port location and the 



sensor array 322 location. In this arrangement, a reflec- 
tor may be utilized to allow multiple ports. For example, 
light, that may travel along light path 327, is admitted to 
the housing of CPU 398 so as to be utilized by sensor 
5 unit 120. 

[0241] A conventional example of CPU 398 for which 
the instant imaging mouse arrangement may be utilized 
with, for example, is an AMiTY VP hand held or pen 
based computer device. Other examples include the 
AMiTY VPN, and the AMiTY XP hand held computer de- 
vices. Other alternate arrangements include, but are not 
limited to, other hand held or pen based computers, and 
also any laptop or other portable computer. Also, for ex- 
ample, hand held devices such as the PALM PI LOT se- 
ries, and other PDA type devices, may be utilized in the 
practice of the instant invention.. 
[0242] In the arrangement illustrated in Figure 8, the 
imaging mouse device 1000 may analogously provide 
any or all of the functions, operating modes, and capa- 
bilities as described previously for Figures 1 through 7. 
In one arrangement, the sensory unit 120 and the sen- 
sory array 322 are configured with an appropriate lens 
arrangement, that is not shown, so as to provide the vid- 
eo camera mode for the user. For example, the hand 
held computer may be propped up, or provided with legs 
folding out sim ilar to that described for the mouse 1 00D 
in Figure 3D, so that a tripod stance may be formed. 
Also, while CPU 398 utilizes the video camera mode, 
preferably the user is also able to utilize the handheld 
CPU 398 in its mode of a portable computer. 
[0243] Either alternatively, or in addition to the above 
video conferencing, the handheld CPU 398 utilizes the 
alternate imaging unit input port 380 and the sensor ar- 
ray port area 254 so as to be utilized in a similar manner 
to a scanner in a supermarket checkout stand. For ex- 
ample, any machine decodable symbols images, text 
and the like may be imaged and recognized by the CPU 
398. 

[0244] Alternately, the sensory unit 120 and sensor 
array 322 may operate in a manner that appears intuitive 
to the user, preferably in a similar manner to a pressure 
touch pad on a laptop computer. The sensor array 322 
may detect edges of an image of a portion of a hand, e. 
g., a thumb, fingers, or a palm, and translate the relative 
motion of the hand or other object that is placed near 
the sensor array 322 into a relative X-Y displacement, 
i.e., cursor signal. As another example, a pen pointing 
device may utilized with the CPU 398, where the pen 
pointing device may be imaged with or without the hand , 
depending upon, e.g., the focal length setting and the 
proximity of the hand and/or object to the sensor. The 
sensory unit 120 and sensor array 322 may translate 
the relative motion of almost any object into a cursor sig- 
nal, that may be represented as a cursor on the display 
290. 

[0245] Alternatively, the sensor array port 254 may 
provide the input for imaging objects, to obtain the rel- 
ative motion to control the cursor, if desired. 
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[0246] Of course, in Figures 1-7, preferably the 
mouse itself is moved by the user, relative to the area 
to be imaged. However, in the arrangement in Figure 8, 
the sensor unit, e.g., comprising at least a pari of the 
housing, may remain relatively stationary while the area 
to be imaged, e.g., the surface, is moved relative to the 
mouse 1000, by the user. 

[0247] Alt h ou g h n ot s h own , yet anoth e r alte mat i ve ar- 
rangement is similar to the mouse 1000 as illustrated in 
Figure 8, except that the sensor array 322 is relocated 
and further integrated with the alternate imaging unit in- 
put port 380, so that a conventional touch pad or the 
like, although not shown, may be located in the position 
vacated by the sensor array 322 Thus, the relative X-Y 
cursor signal is provided conventionally. Again, in this 
alternate arrangement, only one sensor array may be 
needed to practice the invention. This alternate arrange- 
ment is somewhat analogous to the arrangement as de- 
scribed in Figure 1 A-C, wherein a conventional type X- 
Y relative movement cursor signal generating device is 
utilized in conjunction with the mouse for imaging, for 
example, a machine decodable symbol. 
[0248] In yet another arrangement of the mouse 1 000 
of Figure 8, a second alternate imaging unit input port 
and sensor array port area, although not shown, may 
be provided. For example, the second alternate imaging 
Unit input port and sensor array port may be located on 
a side directly opposite the display. Thus, the handheld 
CPU 398 may, for example, be moved about on a sur- 
face by the user to provide either X-Y location data and/ 
or image data, for example, of bar code symbols and 
the like. 

[0249] Yet another alternate arrangement provides 
for a variable focus lens assembly and/or a "dial-a-lens 1 
integrated arrangement so as to provide either a single 
mode function, or multiple mode functions, as previously 
described for Figures 1-7. 

[0250] Yet another alternate arrangement comprises 
a variable focus lens assembly 1 30 that may be hinged. 
Also, the mouse may be mounted upon a hingeable por- 
tion of the CPU 398 so that the mouse may "pop up' so 
as to be more convenient to utilize in a video camera 
mode, and/or code reader mode. For example, a pre- 
ferred alternate arrangement is utilized with a pop-up 
portion that forms, e.g., a 45 degree angle relative to the 
display surface 290 of the CPU 398 of Figure 8. 
[0251] Any of the devices described herein, including 
any of the devices described with respect to Figure 8, 
may image and store the image data for later use by a 
computer. Alternately, the devices may provide either a 
partial or complete on-board processing of the image 
data, that may be utilized and displayed by the device 
itself, or transmitted to another device or computer for 
utilization and display. 

[0252] As shown in Figure 8, the imaging array need 
not be contained in the housing itself. Instead, a cable 
or other wireless techniques described herein may be 
utilized to allow a handheld device 680 to contain the 



CCD device or CMOS array imager. The handheld de- 
vice 680 may provide the image data either in approxi- 
mate real time, or in storage for later transmission. 
[0253] Finally, yet another alternate embodiment of 
5 the mouse 1000, illustrated in Figure 8, comprises the 
combination of the above described sensory unit 120 
with a touch pad that is not shown. In this arrangement, 
the touch pad does not displace the sensor array 322 
but instead is co-located in the same general position 
with sensory unit 322, such that the sensor unit 1 20 may 
provide for X-Y cursor movement by observing relative 
movement of an object through its field of view. Howev- 
er, a touch pad type portion is provided as a pressure 
sensitive device for determination of relative Z axis data. 
The Z axis is preferably normal to at least one of the 
view screen surface of the CPU 398 and the touch pad. 
The sensor array 322 is controlled such that the issuing 
of X-Y position update commands is stopped when in- 
adequate pressure is realized upon the touch pad area. 
[0254] Alternately, processing may be utilized to 
sense when the object is sufficiently out of focus so as 
to cause the mouse 1000 to stop issuing X-Y position 
update commands. Also alternatively, a pulsed laser or 
strobe LED type arrangement may be utilized particu- 
larly with the alternate imaging Unit input port 380 in the 
alternate sensor array port 234 so as to control the stop 
and start of issuing X-Y position update commands by 
mouse 1000. Other techniques as described herein may 
also be utilized to provide Z axis data. 
[0255] Any of the embodiments of the mouse 100 of 
Figures 1-7, or the mouse 1000 that is incorporated, e. 
g., into each of a PDA and the other portable processing 
devices as described in Figure 8, are understood to be 
the "mouse" as used in this Application. 
[0256] The mouse may also be utilized remotely for 
later downloading. 

[0257] For example, a wireless battery operated 
mouse may be utilized remote from a host, e.g., a CPU, 
to collect one or more sets of data, e.g., machine deco- 
dable symbols. The symbols may be stored onboard the 
mouse, for later processing, or may be processed on- 
board and then stored. In other words, the data in this 
example, a barcode symbol, may be stored in either an 
undecoded or a decoded state, respectively. At a later 
time, the mouse may be either manually triggered, or 
automatically triggered, e.g., by being within communi- 
cation range of a desired host, and then may download 
the data. 

[0258] This arrangement may be particularly desira- 
ble for a user utilizing a shopping type of application, e. 
g., such as a home shopping activity that may be found 
on radio, television, the internet, or via other communi- 
cative links. The user utilizes the mouse device in one 
or more of its modes to scan items, e.g. in a kitchen. 
Items of user scanning interest may include, for exam- 
ple, the contents of a refrigerator, such as the barcode 
symbols on each of the items in the refrigerator. Alter- 
nately, or additionally, a video of the contents of the, e. 
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g. p refrigerator, may be imaged and stored for later 
download to the host. Alternately or additionally the us- 
er may scan barcodes associated with a list of items de- 
sired to be ordered. In this example, when physically 
moved into the kitchen, the mouse is not within commu- 
nication range of a desired host. Thus, the mouse may 
process the symbol fully, and then store the decoded 
result in its memory. When the user places the mouse 
back within wireless communication range of the host, 
the host receives the decoded data from the mouse, and 
may automatically further utilize the data. For example, 
the decoded data may be utilized to order items from 
the shopping application. This further utilization by the 
host may be user defined, based upon user preferenc- 
es. 

[0259] It will be understood by one skilled in the art, 
that all of the alternate physical and logical arrange- 
ments, as described in relation to Figures 1-7, may be 
selectively applied, as desired, with the arrangements 
as described for Figure 8, and vice versa. 
[0260] The invention has been described in reference 
to particular embodiments as set forth above. However, 
only the preferred embodiments of the present inven- 
tion, and but a few examples of its versatility are shown 
and described in the present disclosure. It is to be un- 
derstood that the present invention is capable of use in 
various other combinations and environments, and is 
capable of changes or modifications within the scope of 
the inventive concept as expressed herein. Also, many 
modifications and alternatives will become apparent to 
one of skill in the art without departing from the principles 
of the invention as defined by the appended claims. 



Claims 

1. An apparatus comprising: 
a housing, 

an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a surface, 

a control unit coupled to the imaging unit and 
receiving the electrical signals therefrom, the 
control unit processing the electrical signals 
and providing a control signal as an output 
thereof, the control signal provided in response 
to movement of an image of the surface as at 
least one of the housing and the surface move 
relative to one another, 

the control unit is further operable to process 
the electrical signals for detecting a symbol on 
the surface and for generating a code indication 
signal in response thereto, 
the apparatus adapted for transmitting the con- 
trol signal to a computer for controlling a cursor 
of the computer in response to the relative 
movement, and for transmitting the code indi- 



cation signal to the computer. 

2. The apparatus as recited in claim 1 , wherein the 
housing moves over the surface, and 

5 

the imaging unit incorporates no mechanical el- 
ements that move in response to a translational 
movement of the housing across the surface. 

10 3. The apparatus as recited in claim 1 , 

the control unit generates the code indication 
signal when the control signal indicates that the 
housing is positioned relative to the surface in 
an area to be sensed. 

4. The apparatus as recited in claim 1 , wherein 

the symbol comprises one of a bar code, a 
20 stacked bar code, a matrix code, and a machine 

decodable image. 

5. The apparatus as recited in claim 1 , wherein 

25 the control unit is operable in a first mode for 

generating the control signal, and 
the control unit is further operable in a second 
mode to process the electrical signals for de- 
tecting an image of a three dimensional scene- 

30 and generating a video signal in response 

thereto, 

the apparatus further adapted for transmitting 
the video signal to the computer. 

3S 6. The apparatus as recited in claim 5, wherein 

the housing is adapted so as to be utilized as a 
video camera. 

40 7. The apparatus as recited in claim 1, further com- 
prising: 

an optical lens, 

wherein the optical lens cooperates with the im- 
45 aging unit to provide light to the imaging unit, 

and wherein 

the control unit processes the output from the 
imaging unit, so as to provide a video signal out- 
put, such that the housing may be utilized as a 
so video camera. 

8. The apparatus as recited in claim 1 , wherein 

the housing allows for ambient illumination of 
55 at least a part of the surface. 

9. The apparatus as recited in claim 1 , wherein 
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the housing contains a light source for illumi- 
nating the image. 

10. The apparatus as recited in claim 1 , -further com- 
prising: 

a light path modifier mounted within the hous- 
ing, wherein 

the light path modifier is positionable to direct 
light from the surface toward the imaging unit. 

11. The apparatus as recited in claim 10, wherein 

the light path modifier comprises a reflector. 

12. The apparatus as recited in claim 1 , wherein 

the housing contains the imaging unit, and 
wherein 

the housing further comprises a first light -trans- 
missive window and a second light-transmis- 
sive window, the second light- trans missive win- 
dow substantially perpendicular to the first light 
transmissive window, and wherein 
a beam splitter is mounted to the housing to di- 
rect a portion of the light from the surface, after 
the light passes through the first light-transmis- 
sive window, to the imaging unit, and 
the beam splitter transmitting a remaining por- 
tion of the light to pass through the second light- 
transmissive window for viewing by an operator 
of the apparatus. 

13. The apparatus as recited in claim 1 , wherein 

the apparatus generates an illumination of at 
least a part of the surface by providing at least 
one of a strobe-light and a pulsed laser. 

14. The apparatus as recited in claim 13, wherein 

the strobe-light comprises a synchronized ar- 
ray of LED's. 

15. The apparatus as recited in claim 1 , further com- 
prising: 

a light source coupled with the control unit for 
illuminating the surface, 
a viewfinder mounted to the housing for provid- 
ing a view of an area being scanned by the im- 
aging unit. 

16. The apparatus as recited in claim 1 , wherein 

the control unit is operable in a first mode for 

generating the control signal, and 

the control unit is further operable in a second 



mode to process the electrical signals for de- 
tecting a two-dimensional symbol and for gen- 
erating a two-dimensional signal in response 
thereto, 

s the apparatus adapted for transmitting the con- 

trol signal to the computer for controlling a cur- 
sor of the computer in response to the relative 
movement, and for transmitting the two-dimen- 
sional signal to the computer. 

10 

17. The apparatus as recited in claim 1 6, wherein 

the two-dimensional symbol is electronically 
painted, and the electronic painting is dis- 
*5 played. 

18. The apparatus as recited in claim 1 , wherein 

the imaging unit is responsive to an output 
from the control unit, the control unit comprising: 

20 

an image processor responsive to the electrical 
signals from the imaging unit for providing proc- 
essor data at an output thereof, 
a memory coupled with the image processor, 
25 the image processor utilizing the memory for 

storage and retrieval of processor data, and 
a controller responsive to the processor data 
for generating the control signal. 

30 19. An apparatus as recited in claim 18, wherein 

the controller comprises a USB controller. 

20. An apparatus as recited in claim 1 , wherein 

35 

the imaging Unit comprises a charge coupled 
device. 

21 . An apparatus as recited in claim 1 , wherein 

40 

the imaging unit comprises a CMOS imaging 
array. 

22. An apparatus as recited in claim 1 , wherein 

45 

the imaging unit provides the electrical signals 
indicative of an image of material fibers in the 
surface. 

so 23. An apparatus as recited in claim 1 , wherein 

the housing comprises a computer mouse. 

24. An apparatus as recited in claim 1, wherein the 
55 housing comprises a portable data processing de- 
vice. 

25. An apparatus as recited in claim 24 wherein the 
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portable data processing device comprises one of 
a laptop computer, a handheld computer, and a 
PDA. 

26. An apparatus as recited in claim 1 , wherein 

the control unit comprises a processor, and 
wherein 

the processor utilizes a self-learning mode in 
processing the electrical signals. 

27. A method of sensing relative movement and sym- 
bols utilizing a computer mouse, comprising the 
steps of: 

sensing light from a surface containing a sym- 
bol, the sensing performed by an imaging unit, 
generating electrical signals in response to the 
sensed light, 

processing the electrical signals, for generating 
a control signal indicative of relative movement 
of the imaging unit and the surface, and for gen- 
erating code indication signals corresponding 
to the symbol, 

transmitting the control signal to a computer for 
controlling a cursor of the computer, and 
transmitting the code indication signals for fur- 
ther processing by the computer. 

28. A method as recited in claim 27, further comprising 
the step of: 

illuminating at least a part of the surface. 

29. A method as recited in claim 28, wherein 

illuminating at least a part of the surface com- 
prises allowing ambient light to reach the sur- 
face. 

30. A method as recited in claim 27, further comprising 
the step of: 

optically directing the light from the surface, to- 
wards the imaging unit prior to sensing. 

31. A method as recited in claim 30, wherein 

the step of optically directing the light compris- 
es changing a focal length property of a lens 
system. 

32. A method as recited in claim 27, further comprising 
the step of: 

manually initiating a trigger state to condition 
the generating of the electrical signals. 



33. A method as recited in claim 27, further comprising 
the step of 

manually initiating a trigger state to condition 
5 the processing of the electrical signals. 

34. A method as recited in claim 27, further comprising 
the step of: 

moving the mouse over the surface, the surface 
remaining relatively stationary. 

35. A method as recited in claim 27, wherein 

the symbol comprises one of a bar code, a 
stacked bar code, a matrix code, and a machine 
decodable image. 

36. A computer mouse apparatus comprising: 

a housing moveable across a surface, 
an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of the surface, the imaging unit comprising 
one of a CMOS array and a CCD device, 
a control unit coupled to the imaging unit and 
receiving the electrical signals therefrom, the 
control unit processing the electrical signals 
and providing image signals as an output there- 
of, 

an incremental encoder, providing a control sig- 
nal as an output thereof, in response to me- 
chanical movement of an encoder element, as 
the mouse moves across the surface, 
the mouse adapted for transmitting the control 
signal from the control unit output to a computer 
for controlling a cursor of a display, and 
the mouse further adapted for transmitting the 
image signals from the control unit output to the 
computer. 

37. The computer mouse apparatus as recited in claim 
36, wherein 

the control unit is further operable to process 
the electrical signals for detecting a symbol on 
the surface and for generating a code indication 
signal in response thereto, wherein 
the symbol comprises a machine decodable 
image. 

38. A computer mouse apparatus as recited in claim 36, 
wherein 

the control unit comprises a USB controller. 

39. A method for imaging with a mouse, comprising 
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a. imaging an area to be sensed, 

b. generating a first image signal representing 
the area to be sensed, 

c. generating a first position signal representa- 
tive of a location of the first image signal, 

d. generating a second image signal, 

e. generating a second position signal repre- 
sentative of a location of the second image sig- 
nal, 

f. generating an assembled image signal by uti- 
lizing the first and second image signals and 
the first and second position signals. 

40. A method for imaging with a mouse as recited in 
claim 39, further comprising the steps of 

generating a code indication signal in response 
to the assembled image signal, 
providing an output adaptable for transmitting 
the code indication signal to a computer. 

41. A method for imaging with a mouse as recited in 
claim 39, further comprising the steps of: 

processing the assembled image signal for de- 
tecting a symbol on the surface, 
generating a code indication signal in response 
thereto, 

providing an output adaptable for transmitting 
the code indication signal to a computer. 

42. A method for imaging with a mouse as recited in 
claim 41 , wherein 



claim 44, wherein 

the assembled image and the void are graphi- 
cally represented on a computer display device 
utilizing at least one of 
contrasting colors, and 
contrasting gray scales, respectively. 

47. A method for imaging with a mouse as recited in 
claim 44, wherein 

the displaying the partially assembled image 
utilizes at least one of 
a thumbnail sketch view, 
a window view, and 
a full screen view. 

48. A method for detecting a non-translational move- 
ment of a computer mouse, comprising the steps of: 

a. generating a first and second light levels as 
an output, the output incident on at least a por- 
tion of a surface to be imaged, 

b. sensing reflected light from the surface in 
synchronization with the generation of at least 
one of the first and second light levels , 

c. determining if the sensed light is above a pre- 
determined threshold limit, and if so, 

i. generating atranslational position update 
signal as an output, and if not, 

ii. preventing the generation of the trans la- 
tional position update signal as an output, 
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the symbol comprises a two-dimensional ma- 3S 
chine decodable image. 



43. A method for imaging with a mouse as recited in 
claim 39, further comprising the step of: 

40 

displaying an approximate real-time image cor- 
responding to the assembled image signal gen- 
eration. 

44. A method for imaging with a mouse as recited in 45 
claim 43, further comprising the step of: 

displaying a partially assembled image com- 
prising at least one void. 

so 

45. A method for imaging with a mouse as recited in 
claim 44, further comprising the step of: 

moving the mouse to position a displayed cur- 
sor over the at least one void to image the at 
least one void. 

46. A method for imaging with a mouse as recited in 



wherein the translational position update 
signals are adaptable for controll ing a cu rsor on 
the computer. 

49. A method for detecting a non-translational move- 
ment of a computer mouse, comprising the steps of: 

a. imaging at least a portion of a surface to be 
imaged, 

b. generating an image signal representative of 
at least a portion of the area to be imaged, 

c. processing the image signal to determine 
whether the mouse is within a predetermined 
threshold focal length from at least a portion of 
the area to be imaged, and wherein, if so, the 
computer mouse 

i. generates a translational position update 
signal as an output, and if not, 

ii. withholds the generation of atranslation- 
al position update signal as an output. 

50. A method for detecting a non-translational move- 
ment of a computer mouse, as recited in claim 49, 
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wherein the translational position update signals 
are adaptable lor controlling a cursor on the com- 
puter. 

51 . A method lor detecting a non -translation a I move- 
ment ol a computer mouse, comprising the steps of: 

a. generating a first signal in response to a first 
condition of a contact sensing device, 

b. generating a second signal in response to a 
second condition of a contact sensing device, 
wherein the computer mouse 

i. generates a translational position update 
signal as an output in response to the first 
signal, and 

ii. prevents the generation of the transla- 
tional position update signal as an output 
in response to the second signal. 

52. A method for detecting a non-translational move- 
ment of a mouse as recited in claim 51 , wherein 

the contact sensing device further comprises at 
least one micros witch. 



an incremental encoder, providing a control sig- 
nal as an output thereof, the control signal in 
response to mechanical movement of an en- 
coder element as the mouse moves across the 

5 surface, 

the mouse adapted for transmitting the control 
signal from the control unit output to a computer 
for controlling a cursor of a display, and, 
the mouse further adapted for transmitting the 

10 electrical signals from the controller output to 

the computer for processing. 

57. The computer mouse apparatus as recited in claim 
56, wherein 

is 

the imaging unit comprises one of a CMOS ar- 
ray and a CCD device, 

the image is a symbol comprising one of a bar 
code, a stacked bar code, matrix code, a ma- 
£0 chine decodable image, and an alphanumeric 

symbol, and 

the image signal processing comprises decod- 
ing the symbol. 

25 58. A portable computer apparatus comprising: 



53. A method for detecting a non-translational move- 
ment of mouse as recited in claim 51 , wherein 

the contact sensing device further comprises at 
least one physical pressure responsive device. 

54. A computer mouse comprising: 

a housing, 

an imaging unit attached to the housing and 
providing image signals indicative of an image 
of a surface, the surface containing a symbol, 
the mouse adapted for transmitting the image 
signals to a computer for controll ing a cursor of 
the computer in response to relative movement 
between the mouse and the surface, and for 
processing the image signals to decode the 
symbol. 

55. The computer mouse as recited in claim 54, wherein 

the symbol is one of a bar code, a stacked bar 
code, a matrix code, and a machine decodable 
image. 

56. A computer mouse apparatus comprising: 

a housing movable across a surface, 
an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of the surface, the surface containing a 
symbol, 



a portable laptop computer having a housing, 
wherein at least one of the housing and a sur- 
face are movable relative to each other, 
30 an imaging unit attached to the housing and 

providing electrical signals indicative of an im- 
age of a symbol on the su rface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

3S a control unit coupled to the imaging unit and 

receiving the electrical signals therefrom, the 
control unit decoding the electrical signals and 
providing data signals to the laptop computer. 

40 59. a portable computer apparatus comprising: 

a portable handheld computer having a hous- 
ing, wherein at least one of the housing and a 
surface are movable relative to each other, 
45 an imaging unit attached to the housing and 

providing electrical signals indicative of an im- 
age of a symbol on the su rface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

so a control unit coupled to the imaging unit and 

receiving the electrical signals therefrom, the 
control unit decoding the electrical signals and 
providing data signals to the handheld compu- 
ter. 

55 

60. A portable computer apparatus comprising: 

a pen based computer having a housing, 
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wherein at least one of the housing and a sur- 
face are movable relative to each other, 
an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a symbol on the surface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

a control unit coupled to the imaging unit and 
receiving the electrical signals therefrom, the 
control Unit decoding the electrical signals and 
providing data signals to the pen based com- 
puter. 

61. A portable apparatus comprising: 



a portable laptop computer having a housing, 
wherein at least one of the housing and a sur- 
face are movable relative to each other, 
an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a symbol on the surface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

the laptop computer coupled to the imaging unit 
and receiving the electrical signals therefrom, 
for performing at least one of decoding and 
storing the electrical signals. 

63. A portable computer apparatus comprising: 

a pen based computer having a housing, 
wherein at least one of the housing and a sur- 
face are movable relative to each other, 
an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a symbol on the surface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

the pen based computer coupled to the imaging 
unit and receiving the electrical signals there- 
from, for performing at least one of decoding 
and storing the electrical signals. 



64. A portable apparatus comprising: 

a PDA having a housing, wherein at least one 
of the housing and a surface are movable rela- 

5 five to each other, 

an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a symbol on the su rface, the imaging unit 
incorporating one of a CMOS array and CCD 

10 device, and 

the PDA coupled tothe imaging unit and receiv- 
ing the electrical signals therefrom, for perform- 
ing at least one of decoding and storing the 
electrical signals. 

is 

65. An apparatus comprising: 
a housing, 

an imaging unit attached to the housing and 
providing electrical signals indicative of an im- 
age of a surface, the imaging unit comprising 
one of a CMOS array and a CCD device, 
a control unit coupled to the imaging unit and 
receiving the electrical signals therefrom, the 
control unit processing the electrical signals for 
detecting a symbol on the surface and provid- 
ing a code indication signal as an output there- 
of, 

the apparatus further adapted for transmitting 
the code indication signals from the control unit 
as an output. 

66. The apparatus as recited in claim 65, wherein 

3S the apparatus is further adapted for transmit- 

ting the code indication signals from the control 
unit output to a computer. 

67. The computer mouse apparatus as recited in claim 
40 65, wherein 

the control unit is further operable to process 
the electrical signals for generating image sig- 
nals as an output thereof; and wherein 
45 the symbol comprises a machine decodable 

image. 

68. A computer mouse apparatus as recited in claim 65, 
wherein 

so 

the control unit comprises a USB controller. 

69. A computer mouse apparatus as recited in claim 65, 
wherein 

55 

the imaging unit comprises no moving parts. 



a PDA having a housing, wherein at least one 
of the housing and a surface are movable rela- 
tive to each other, 

an imaging unit attached to the housing and 
providing electrical signals indicative of an im- so 
age of a symbol on the surface, the imaging unit 
incorporating one of a CMOS array and CCD 
device, and 

a control unit coupled to the imaging unit and 
receiving the electrical signals therefrom, the 25 
control unit decoding the electrical signals and 
providing data signals to the PDA. 

62. A portable computer apparatus comprising: 
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FIG. 7A 
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FIG. 7B 
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